CN116095308A

CN116095308A - Image decoding method and device

Info

Publication number: CN116095308A
Application number: CN202111335705.5A
Authority: CN
Inventors: 孙煜程; 曹小强; 陈方栋; 王莉
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2021-11-11
Filing date: 2021-11-11
Publication date: 2023-05-09
Also published as: CN116389722A

Abstract

The embodiment of the application provides an image decoding method and device, relates to the field of image encoding and decoding, and can improve image decoding performance. The method comprises the following steps: analyzing the code stream to obtain a prediction mode of the current prediction unit and a residual error value between a true value and a predicted value of the current prediction unit; the prediction mode of the current prediction unit at least comprises a first prediction mode and a second prediction mode, wherein the first prediction mode is used for predicting a first part of the current prediction unit, the second prediction mode is used for predicting a second part of the current prediction unit, and the first prediction mode is different from the second prediction mode; predicting a first part of the current prediction unit by adopting a first prediction mode, and predicting a second part of the current prediction unit by adopting a second prediction mode to obtain a predicted value of the current prediction unit; and determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

Description

Image decoding method and device

Technical Field

The embodiment of the application relates to the technical field of image encoding and decoding, in particular to an image decoding method and device.

Background

With the rapid development of video/image services, the data volume of video/images is increased, a large amount of transmission bandwidth is occupied for transmitting video/images, and a large storage space is also required for storing video/images.

Taking an image as an example, an image encoder is used for encoding the image to obtain a code stream and transmitting the code stream, and then an image decoder is used for analyzing the code stream to obtain the image. Taking a coding block as an example, the image coding process mainly includes predicting the coding block, calculating a residual value between a predicted value of the coding block and a true value of the coding block, transforming the residual value, quantizing a transformation result, and coding a quantization result, thereby obtaining a code stream. The image decoding process mainly comprises the steps of predicting a coding block, decoding a code stream, performing inverse quantization on a decoding result, performing inverse transformation on an inverse quantization result, and calculating the sum of a predicted value of a coding block element and a residual value obtained by inverse transformation so as to obtain a reconstruction value of the coding block.

In the image decoding process, the prediction result of the coding block has a great influence on the compression efficiency of the image, and at present, the existing method for predicting the coding block still has low compression efficiency, so that the image decoding performance is not high.

Disclosure of Invention

The embodiment of the application provides an image decoding method and device, which can improve image decoding performance.

In order to achieve the above purpose, the embodiment of the application adopts the following technical scheme:

in a first aspect, an embodiment of the present application provides an image decoding method, including: analyzing the code stream to obtain a prediction mode of the current prediction unit and a residual error value between a true value and a predicted value of the current prediction unit; the prediction mode of the current prediction unit at least comprises a first prediction mode and a second prediction mode, wherein the first prediction mode is used for predicting a first part of the current prediction unit, the second prediction mode is used for predicting a second part of the current prediction unit, and the first prediction mode is different from the second prediction mode; then, predicting a first part of the current prediction unit by adopting a first prediction mode, and predicting a second part of the current prediction unit by adopting a second prediction mode to obtain a predicted value of the current prediction unit; and determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual value of the current prediction unit.

In the image decoding method provided by the embodiment of the application, the decoding equipment predicts the first part of the current unit to be predicted by adopting the first prediction mode, adopts the second prediction mode for the second part of the current unit to be predicted, and compared with one prediction mode corresponding to one prediction unit in the prior art, the technical scheme provided by the embodiment of the application divides pixels in the prediction unit, and pixels in different parts adopt different prediction modes, so that the decoding mode is more flexible, and a proper prediction mode is selected according to the characteristics of the pixels in the prediction unit, so that the prediction result is more accurate (namely, is more close to a true value), and the image decoding performance can be improved.

In one possible implementation, pixels in the first portion of the current prediction unit are predicted based on reconstructed pixels outside of the current prediction unit. That is, when predicting a certain pixel in the first portion, it is not necessary to rely on the reconstructed pixels within the current prediction unit (including the first portion), i.e., the prediction process of all pixels in the first portion of the current prediction unit is independent of each other. In this case all pixels in the first part may be predicted in parallel using a first prediction mode, i.e. the pixels of the first part are predicted as a group, which parallel prediction mode corresponds to the first prediction mode, which may also be referred to as group prediction mode.

Pixels in the second portion of the current prediction unit are predicted based on reconstructed pixels outside of the current prediction unit and reconstructed pixels within the current prediction unit, or pixels in the second portion of the current prediction unit are predicted based on reconstructed pixels within the current prediction unit. That is, when a certain pixel in the second portion is predicted, it is necessary to rely on a reconstructed pixel outside the current prediction unit and a reconstructed pixel within the current prediction unit, or when a certain pixel in the second portion is predicted, it is necessary to rely on a reconstructed pixel within the current prediction unit, that is, the prediction process of all pixels in the second portion of the current prediction unit may depend on each other. In this case the pixels in the second part may be predicted pixel by pixel using a second prediction mode, which pixel by pixel prediction mode corresponds to the second prediction mode and may also be referred to as point prediction mode.

In a possible implementation, the pixels in the second part of the current prediction unit are predicted based on reconstructed pixels outside the current prediction unit and reconstructed pixels within the current prediction unit, or the pixels in the second part of the current prediction unit are predicted based on reconstructed pixels within the current prediction unit; the reconstructed pixels within the current prediction unit include reconstructed pixels in the first portion. That is, the prediction of pixels in the second portion needs to rely on reconstructed pixels in the first portion of the current prediction unit.

In a possible implementation manner, the current prediction unit may further include at least one other portion other than the first portion and the second portion, for example, the current prediction unit further includes a third portion, where the first portion, the second portion, and the third portion do not overlap each other.

In a possible implementation manner, the pixels in the first portion of the current prediction unit are predicted based on the reconstructed pixels outside the current prediction unit, and the pixels in the second portion of the current prediction unit are predicted based on the reconstructed pixels outside the current prediction unit, where all the pixels in the first portion are predicted in parallel in a first prediction mode, i.e. the pixels in the first portion are predicted as a group, and the parallel prediction mode corresponds to the first prediction mode and is a group prediction mode; all pixels in the second part are predicted in parallel by adopting the second prediction mode, namely the pixels in the second part can also be predicted as a group, and the parallel prediction mode corresponds to the second prediction mode and is a group prediction mode. It should be noted that, the first prediction mode and the second prediction mode may be both angle prediction modes, but the first prediction mode and the second prediction mode are different angle prediction modes, for example, the first prediction mode is a vertical angle prediction mode, and the second prediction mode is a horizontal angle prediction mode.

In a possible implementation manner, the portion of the current prediction unit except for the first line is divided to obtain a first portion and a second portion, where the first line of the current prediction unit may be encoded according to an original value of the first line, the first portion is predicted by using the first prediction mode (i.e. the group prediction mode), and the second portion is predicted by using the second prediction mode (i.e. the point prediction mode).

In a possible implementation manner, the portions of the current prediction unit except the first row and the first column are divided to obtain a first portion and a second portion, where the first row and the first column of the current prediction unit are encoded with original values, the first portion is predicted with the first prediction mode (i.e. the group prediction mode), and the second portion is predicted with the second prediction mode (i.e. the point prediction mode).

In a possible implementation manner, the portion of the current prediction unit except for the pixel at the upper left corner is divided to obtain a first portion and a second portion, where the pixel at the upper left corner of the current prediction unit is encoded with an original value, the first portion is predicted using the first prediction mode (i.e., the group prediction mode), and the second portion is predicted using the second prediction mode (i.e., the point prediction mode).

In a possible implementation manner, the current prediction unit is divided into three parts, namely a first part, a second part and a third part, wherein different parts can be predicted by adopting the first prediction mode and/or the second prediction mode, namely different combinations of the group prediction mode and the point prediction mode. For example, a first portion is predicted using a vertical angle prediction mode (belonging to the first prediction mode), a second portion is predicted using a horizontal angle prediction mode (belonging to the first prediction mode), and a third portion is predicted using a point prediction mode.

In one possible implementation manner, the parsing the code stream to obtain the residual value between the actual value and the predicted value of the current prediction unit specifically includes: decoding the code stream of the current prediction unit to obtain a quantization coefficient of the current prediction unit; performing inverse quantization processing on the quantized coefficient of the current prediction unit to obtain an inverse quantized coefficient of the current prediction unit; performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the predicted value and the true value of the current prediction unit; when prediction is performed in the vertical direction, horizontal inverse transformation is performed in parallel, and when prediction is performed in the horizontal direction, vertical inverse transformation is performed in parallel, that is, the preset inverse transformation direction is perpendicular to the prediction direction of the current prediction unit, and the current prediction direction may be understood as a prediction angle, for example, the prediction direction may be a horizontal direction, a vertical direction, or a direction forming an arbitrary angle with the reference direction.

In the embodiment of the present application, pixels along a prediction direction are interdependent in a process of predicting the pixels, so that when a predicted value along the prediction direction is transformed, pixels along the prediction direction are interdependent, that is, a plurality of pixels cannot be transformed in parallel along the prediction direction. Since the pixels are mutually dependent in the direction perpendicular to the prediction direction, a plurality of pixels can be converted in parallel in the direction perpendicular to the prediction direction, and thus the decoding efficiency can be improved.

It will be appreciated that in the image encoding process, an image to be encoded is generally divided into a plurality of image blocks, the image blocks are predicted one by one, and then the image blocks are encoded to obtain a code stream, where the image blocks in the image to be encoded may also be referred to as encoding blocks.

In an example, in the embodiment of the present application, the current prediction unit may be a coding block in the image to be coded, and the current prediction unit may also be one prediction unit in a plurality of prediction units obtained by dividing a coding block in the image to be coded, which is specifically determined according to an actual situation, and the embodiment of the present application is not limited.

In a possible implementation manner, the image decoding method provided in the embodiment of the application further includes: dividing the current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block, wherein the current prediction unit is any one of the plurality of prediction units. The preset dividing manner may be any one or a combination of multiple existing dividing methods, specifically selected according to actual requirements, for example, may include a vertical dividing manner, a horizontal dividing manner, a serial grouping manner, and the like, specifically selected according to actual requirements, which is not limited in the embodiments of the present application.

In a second aspect, an embodiment of the present application provides an image decoding method, including: analyzing the code stream to obtain a prediction mode of the current prediction unit and a residual error value between a predicted value and a true value of the current prediction unit; the prediction mode of the current prediction unit is an angle prediction mode, the angle prediction mode meets the preset condition, the preset condition is that the number of pixels |M| of the predicted pixel point and the current pixel point which are determined according to the prediction angle and are spaced in the image height direction is an integer, the number of pixels |N| of the predicted pixel point and the current pixel point which are determined according to the prediction angle and are spaced in the image width direction is an integer, and |M| or |N| is the minimum value in the acceptable integer values, and|represents taking absolute values; predicting the current prediction unit by adopting a prediction mode of the current prediction unit to obtain a prediction value of the current prediction unit; and determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual value of the current prediction unit.

According to the image decoding method provided by the embodiment of the application, the decoding device can predict the current prediction unit by adopting the angle prediction mode, and the angle prediction mode meets the following conditions: the number of pixels, which are determined according to the prediction angle, of the pixels, which are spaced between the prediction pixel point and the current pixel point in the image height direction, is an integer, and the number of pixels, which are spaced between the prediction pixel point and the current pixel point in the image width direction, determined according to the prediction angle, is an integer, namely, the reconstructed pixel value can be directly used as the prediction value of the pixel to be predicted, so that pixel interpolation operation can be avoided, the calculated amount can be reduced, the image decoding efficiency can be improved, and the image decoding performance can be improved.

In one possible implementation of the method, the method comprises,

n is an integer.

In the embodiment of the application, when M and N also meet the following conditions

In the process of predicting the current prediction unit, the original complex multiplication and division operation is changed into simple shift operation, so that the calculation complexity can be reduced.

In one implementation manner, when the current prediction unit is predicted according to the angle prediction mode provided by the embodiment of the application, the current prediction unit can be predicted by adopting the pixels reconstructed in the first two rows of the current prediction unit, and compared with the scheme of predicting the pixel value of the current position according to the pixels in the first row of the current prediction position in the prior art, the embodiment of the application can avoid pixel interpolation operation when the current prediction unit is predicted, so that the calculated amount can be reduced, and the image coding efficiency is improved.

In one possible implementation manner, the parsing the code stream to obtain a residual value between the predicted value and the true value of the current prediction unit specifically includes: decoding the code stream of the current prediction unit to obtain a quantization coefficient of the current prediction unit; performing inverse quantization processing on the quantized coefficient of the current prediction unit to obtain an inverse quantized coefficient of the current prediction unit; performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the predicted value and the true value of the current prediction unit; when prediction is performed in the vertical direction, horizontal inverse transformation is performed in parallel, and when prediction is performed in the horizontal direction, vertical inverse transformation is performed in parallel, that is, the preset inverse transformation direction is perpendicular to the prediction direction of the current prediction unit.

In a possible implementation manner, the image decoding method provided in the embodiment of the application further includes: dividing the current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block, wherein the current prediction unit is any one of the plurality of prediction units.

In a third aspect, an embodiment of the present application provides an image decoding apparatus, including an parsing module, a prediction module, and a reconstruction module. The analysis module is used for analyzing the code stream and acquiring a prediction mode of the current prediction unit and a residual error value between a true value and a predicted value of the current prediction unit; the prediction mode of the current prediction unit at least comprises a first prediction mode and a second prediction mode, wherein the first prediction mode is used for predicting a first part of the current prediction unit, the second prediction mode is used for predicting a second part of the current prediction unit, and the first prediction mode is different from the second prediction mode; the prediction module is used for predicting a first part of the current prediction unit by adopting a first prediction mode, and predicting a second part of the current prediction unit by adopting a second prediction mode to obtain a prediction value of the current prediction unit; the reconstruction module is used for determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

In a fourth aspect, an embodiment of the present application provides an image decoding apparatus, including an parsing module, a prediction module, and a reconstruction module. The analysis module is used for analyzing the code stream and acquiring a prediction mode of the current prediction unit and a residual error value between a predicted value and a true value of the current prediction unit; the prediction mode of the current prediction unit is an angle prediction mode, the angle prediction mode meets preset conditions, the preset conditions are that the number of pixels |M| of the predicted pixel point and the current pixel point which are determined according to the prediction angle and are spaced in the image height direction is an integer, the number of pixels |N| of the predicted pixel point and the current pixel point which are determined according to the prediction angle and are spaced in the image width direction is an integer, and |M| or |N| is the minimum value in the acceptable integer values, and || represents taking absolute values; the prediction module is used for predicting the current prediction unit by adopting a prediction mode of the current prediction unit to obtain a prediction value of the current prediction unit; the reconstruction module is used for determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

In a fifth aspect, embodiments of the present application provide an image decoder configured to perform the image decoding method according to any one of the first aspect and the second aspect and possible implementations thereof.

In a sixth aspect, an embodiment of the present application provides an electronic device, including the image decoder, the communication interface, and the memory according to the fifth aspect; the image decoder is configured to perform the image decoding method according to any of the first and second aspects and possible implementations thereof.

In a seventh aspect, embodiments of the present application provide a computer readable storage medium, including computer instructions, which when executed on an electronic device, cause the electronic device to perform the image decoding method according to any one of the above-mentioned first and second aspects and possible implementations thereof.

In an eighth aspect, embodiments of the present application provide a computer program product for performing the image decoding method according to any one of the first and second aspects and possible implementations thereof, when the computer program product is run on a computer.

In a ninth aspect, embodiments of the present application provide a chip including a memory and a processor. The memory is used for storing computer instructions. The processor is configured to invoke and execute the computer instructions from the memory to cause the electronic device to perform the image decoding method according to any of the first aspect and the second aspect and possible implementations thereof.

In a tenth aspect, an embodiment of the present application provides an image encoding and decoding system, including an image encoding device and an image decoding device, where the image decoding device is configured to receive an image code stream from the image encoding device, and perform the image decoding method according to any one of the first aspect and the second aspect and possible implementations thereof to decode the image code stream, so as to obtain a decoded image.

It should be appreciated that the technical solutions of the third aspect to the tenth aspect and the corresponding possible embodiments of the present application may refer to the technical effects of the first aspect to the second aspect and the corresponding possible embodiments thereof, which are not described herein.

Drawings

Fig. 1 is a schematic architecture diagram of a codec system according to an embodiment of the present application;

FIG. 2 is a schematic block diagram of an encoder provided in an embodiment of the present application;

FIG. 3 is a schematic block diagram of a decoder provided in an embodiment of the present application;

fig. 4 is a schematic flow chart of encoding/decoding according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an image encoding method according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram of a coding block partitioning method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an image decoding method according to an embodiment of the present disclosure;

FIG. 8 is a second schematic diagram of an image decoding method according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a first portion and a second portion of a current prediction unit according to an embodiment of the present application;

FIG. 10 is a schematic diagram of predicting a second portion using a second prediction mode according to an embodiment of the present application;

FIG. 11 is a schematic diagram of a prediction direction according to an embodiment of the present disclosure;

FIG. 12 is a schematic diagram of a second image encoding method according to an embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a predicted angle according to an embodiment of the present disclosure;

FIG. 14 is a third schematic diagram of an image decoding method according to an embodiment of the present disclosure;

FIG. 15 is a diagram illustrating a method for decoding an image according to an embodiment of the present disclosure;

FIG. 16 is a schematic diagram of an angle prediction mode according to an embodiment of the present disclosure;

FIG. 17 is a schematic diagram of predicting a current prediction unit using an angle prediction mode according to an embodiment of the present application;

fig. 18 is a schematic structural diagram of an image decoding device according to an embodiment of the present application;

fig. 19 is a second schematic structural diagram of an image decoding device according to an embodiment of the present application.

Detailed Description

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone.

The terms first and second and the like in the description and in the claims of embodiments of the present application are used for distinguishing between different objects and not necessarily for describing a particular sequential order of objects. For example, the first prediction mode and the second prediction mode, etc., are used to distinguish between different prediction modes, rather than to describe a particular order of prediction modes.

In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In the description of the embodiments of the present application, unless otherwise indicated, the meaning of "a plurality" means two or more. For example, the plurality of prediction units means two or more prediction units, and the plurality of encoded blocks means two or more encoded blocks.

The image decoding method provided in the embodiments of the present application may also be used for video decoding, where it should be understood that the video includes a series of images (pictures), and decoding the video is substantially decoding all the images included in the video.

The system architecture to which the embodiments of the present application apply is described below. Referring to fig. 1, a schematic diagram of an architecture of a codec system (which may also be referred to as an encoding and decoding system) 10 to which embodiments of the present application are applied is provided. As shown in fig. 1, the codec system 10 may include a source device 11 and a destination device 12, the source device 11 being used to encode an image, and thus the source device 11 may be referred to as a video encoding apparatus. The destination device 12 is used to decode the encoded image data generated by the source device 11, and thus the destination device 12 may be referred to as a video decoding apparatus.

The source device 11 and the destination device 12 may include various devices including, for example, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, cell phones, televisions, cameras, display devices, digital media players, video game consoles, vehicle mount computers, wireless communication devices, and the like.

In one example, the source device 11 and the destination device 12 in fig. 1 may be two separate devices, or the source device 11 and the destination device 12 may be the same device, i.e., the source device 11 or the corresponding function and the destination device 12 or the corresponding function may be integrated on the same device.

Communication may be performed between the source device 11 and the destination device 12, for example, the destination device 12 may receive encoded image data from the source device 11. In one example, one or more communication media may be included between source device 11 and destination device 12 through which encoded image data is transmitted, which may include routers, switches, base stations, or other devices that facilitate communication from source device 11 to destination device 12.

As shown in fig. 1, the source device 11 includes an encoder 112, and in one example, the source device 11 may further include an image preprocessor 111 and a communication interface 113. The image preprocessor 111 is configured to perform preprocessing on the received image to be encoded, for example, preprocessing performed by the image preprocessor 111 may include truing, color format conversion (e.g., converting from RGB format to YUV format), toning, denoising, and the like. An encoder 112 for receiving the preprocessed image, processing the preprocessed image using an associated prediction mode (e.g., a prediction mode in various embodiments herein), thereby providing encoded image data. In some embodiments, encoder 112 may be used to perform the image encoding process in various embodiments described below. Communication interface 113 may be used to transfer encoded image data to destination device 12 or any other device (e.g., memory) for storage or direct reconstruction, which may be any device for decoding or storage. The communication interface 113 may also package the encoded image data into a suitable format for transmission.

In one example, the image preprocessor 111, the encoder 112, and the communication interface 113 may be hardware components in the source device 11, or may be software programs in the source device 11, which are not limited in this embodiment.

As further shown in fig. 1, destination device 12 includes a decoder 122, and in one example, destination device 12 may also include a communication interface 121, an image post-processor 123. Wherein the communication interface 121 may be used to receive encoded image data from the source device 11 or any other source device, such as a storage device. The communication interface 121 may also decapsulate the data transmitted by the communication interface 113 to obtain encoded image data. Decoder 122 receives encoded image data and outputs decoded image data (also referred to as reconstructed image data or reconstructed image data). In some embodiments, decoder 122 may be used to perform the decoding process described in various embodiments described below.

An image post-processor 123 for performing post-processing on the decoded image data to obtain post-processed image data. The post-processing performed by the image post-processor 123 may include: color format conversion (e.g., from YUV format to RGB format), toning, truing, or resampling, or any other process, may also be used to transmit the post-processed image data to a display device for display.

Similarly, in one example, the communication interface 121, the decoder 122, and the image post-processor 123 may be hardware components in the destination device 12, or may be software programs in the destination device 12, which is not limited in this embodiment.

The structure of the encoder and decoder of fig. 1 will be briefly described.

Referring to fig. 2, fig. 2 shows a schematic block diagram of an example of an encoder 20 for implementing an embodiment of the present application. In fig. 2, the encoder includes a prediction processing unit 201, a residual calculation unit 202, a transform processing unit 203, a quantization unit 204, an encoding unit 205, an inverse quantization unit (may also be referred to as an inverse quantization unit) 206, an inverse transform processing unit (may also be referred to as an inverse transform processing unit 207), a reconstruction unit (or referred to as a reconstruction unit) 208, and a filter unit 209. In one example, encoder 20 may further comprise a buffer for buffering the reconstructed image block output by reconstruction unit 208, and a decoded image buffer for buffering the filtered image block output by filter unit 209.

The input of the encoder 20 is a picture block of a picture (which may be referred to as a picture to be encoded), which may also be referred to as a current picture block or a picture block to be encoded, and a segmentation unit (not shown in fig. 2) for segmenting the picture to be encoded into a plurality of picture blocks may be included in the encoder 20. The encoder 20 is used for block-by-block encoding to complete encoding of the image to be encoded, e.g. performing an encoding process on each image block.

The prediction processing unit 201 is configured to receive or obtain an image block (a current image block to be encoded of a current image to be encoded, which may also be referred to as a current image block, and the image block may be understood as a true value of the image block) and reconstructed image data, and predict the current image block based on relevant data in the reconstructed image data, so as to obtain a predicted block of the current image block. In one example, the prediction processing unit 201 may include an inter prediction unit, an intra prediction unit, and a mode selection unit for selecting an intra prediction mode or an inter prediction mode, performing a prediction process by the intra prediction unit if the intra prediction mode is selected, and performing a prediction process by the inter prediction unit if the inter prediction mode is selected.

The residual calculation unit 202 is configured to calculate a residual between a true value of an image block and a predicted block of the image block, and obtain a residual block, for example, by subtracting a pixel value of the predicted block from a pixel value of the image block.

The transform processing unit 203 is configured to perform a transform, such as a discrete cosine transform (discrete cosine transform, DCT) or a discrete sine transform (discrete sine transform, DST), on the residual block to obtain transform coefficients in a transform domain, which may also be referred to as transform residual coefficients, which may represent the residual block in the transform domain.

The quantization unit 204 is configured to quantize the transform coefficients by applying scalar quantization or vector quantization to obtain quantized transform coefficients, which may also be referred to as quantized residual coefficients. The quantization process may reduce the bit depth associated with some or all of the transform coefficients. For example, n-bit transform coefficients may be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The quantization level may be modified by adjusting quantization parameters (quantization parameter, QP). For example, for scalar quantization, different scales may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The appropriate quantization step size may be indicated by a quantization parameter (quantization parameter, QP).

The encoding unit 205 is configured to encode the quantized residual coefficients, and the encoded image data (i.e. the encoding result of the current image block to be encoded) output in the form of an encoded bitstream, which may then be transmitted to a decoder, or stored, and subsequently transmitted to the decoder or used for retrieval. The encoding unit 205 may also be used to encode other syntax elements of the current image block, such as encoding a prediction mode to a bitstream, etc. The coding algorithms include, but are not limited to, variable length coding (variable length coding, VLC) algorithms, context Adaptive VLC (CAVLC) algorithms, arithmetic coding algorithms, context adaptive binary arithmetic coding (context adaptive binary arithmetic coding, CABAC) algorithms, syntax-based context adaptive binary arithmetic coding (SBAC) algorithms, probability interval partitioning entropy (probability interval partitioning entropy, PIPE) algorithms.

The inverse quantization unit 206 is configured to inverse quantize the quantized coefficient to obtain an inverse quantized coefficient, where the inverse quantization is applied by the quantization unit 204, for example, based on or using the same quantization step size as the quantization unit 204, and an inverse quantization scheme of the quantization scheme applied by the quantization unit 204 is applied. The dequantized coefficients may also be referred to as dequantized residual coefficients.

The inverse transform processing unit 207 is configured to perform an inverse transform on the above-mentioned inverse quantized coefficients, and it should be understood that the inverse transform is an inverse application of the above-mentioned transform processing unit 203, and for example, the inverse transform may include an inverse discrete cosine transform (discrete cosine transform, DCT) or an inverse discrete sine transform (discrete sine transform, DST) to obtain an inverse transform block in a pixel domain (or referred to as a sample domain). The inverse transform block may also be referred to as an inverse transformed inverse quantized block or an inverse transformed residual block.

The reconstruction unit 208 is configured to add an inverse transform block (i.e. an inverse transform residual block) to the prediction block to obtain a reconstructed block in the sample domain, and the reconstruction unit 208 may be a summer, e.g. to add sample values (i.e. pixel values) of the residual block to sample values of the prediction block. The reconstructed block output by the reconstruction unit 208 may subsequently be used for predicting other image blocks, e.g. for use in intra prediction mode.

A filter unit 209 (or simply "filter") is used to filter the reconstructed block to obtain a filtered block, to smooth pixel transitions or to improve image quality. The filter unit may be a loop filter unit intended to represent one or more loop filters, such as deblocking filters, sample-adaptive offset (SAO) filters or other filters, such as bilateral filters, adaptive loop filters (adaptive loop filter, ALF), or sharpening or smoothing filters, or collaborative filters. In one example, the filtered block output by the filtering unit 209 may be subsequently used to predict other image blocks, for example, for use in inter prediction mode.

Specifically, in the present embodiment, the encoder 20 is used to implement the image encoding method described in the later embodiments.

Referring to fig. 3, fig. 3 shows a schematic block diagram of an example of a decoder 30 for implementing an embodiment of the present application. Decoder 30 is operative to receive encoded image data (i.e., an encoded bitstream, e.g., an encoded bitstream including image blocks and associated syntax elements) encoded, for example, by encoder 20, to obtain a decoded image. The decoder 30 includes a decoding unit 301, an inverse quantization unit 302, an inverse transform processing unit 303, a prediction processing unit 304, a reconstruction unit 305, and a filter unit 306. In some examples, decoder 30 may perform a decoding pass that is substantially reciprocal to the encoding pass described by encoder 20 of fig. 2. In one example, decoder 30 may further comprise a buffer for buffering reconstructed image blocks output by reconstruction unit 305, and a decoded image buffer for buffering filtered image blocks output by filter unit 306.

The decoding unit 301 is configured to perform decoding on the encoded image data to obtain quantized coefficients and/or decoded encoding parameters (e.g., the decoding parameters may include any or all of inter-prediction parameters, intra-prediction parameters, filter parameters, and/or other syntax elements). The decoding unit 301 is further configured to forward the decoded coding parameters to the prediction processing unit 304, so that the prediction processing unit performs a prediction process according to the coding parameters.

The function of the inverse quantization unit 302 may be the same as the function of the inverse quantization unit 206 of the encoder 20 for inverse quantizing (i.e., inverse quantizing) the quantized coefficients decoded by the decoding unit 301.

The function of the inverse transform processing unit 303 may be the same as the function of the inverse transform processing unit 207 of the encoder 20, and the function of the reconstruction unit 305 (e.g. the summer) may be the same as the function of the reconstruction unit 208 of the encoder 20, for performing an inverse transform (e.g. an inverse DCT, an inverse integer transform or a conceptually similar inverse transform process) on the quantized coefficients, resulting in an inverse transform block (which may also be referred to as an inverse transform residual block), i.e. a residual block of the current image block in the pixel domain.

The prediction processing unit 304 is configured to receive or obtain encoded image data (e.g., an encoded bitstream of a current image block) and reconstructed image data, and the prediction processing unit 301 may further receive or obtain prediction related parameters and/or information about the selected prediction mode (i.e., decoded encoding parameters) from, for example, the decoding unit 302, and predict the current image block based on the related data and the decoded encoding parameters in the reconstructed image data to obtain a predicted block of the current image block. In one example, the prediction processing unit 304 may include an inter prediction unit, an intra prediction unit, and a mode selection unit for selecting an intra prediction mode or an inter prediction mode, performing a prediction process by the intra prediction unit if the intra prediction mode is selected, and performing a prediction process by the inter prediction unit if the inter prediction mode is selected.

The reconstruction unit 305 is for adding an inverse transform block, i.e. an inverse transform residual block, to the prediction block to obtain a reconstructed block in the sample domain, e.g. adding sample values of the inverse transform residual block to sample values of the prediction block.

The filter unit 306 is configured to filter the reconstructed block to obtain a filtered block, i.e. a decoded image block.

Specifically, in the present embodiment, the decoder 30 is used to implement the image decoding method described in the later embodiments.

It should be understood that, in the encoder 20 and the decoder 30 according to the embodiments of the present application, the processing result for a certain link may be further processed and then output to a next link, for example, after the links such as interpolation filtering, motion vector derivation or filtering, the processing result for the corresponding link may be further processed by performing operations such as Clip or shift.

On the basis of the codec system, the encoder and the decoder shown in fig. 1 to 3, the embodiments of the present application provide a possible codec implementation, as shown in fig. 4, fig. 4 is a schematic flow chart of a codec provided in the embodiments of the present application, where the codec implementation includes processes (1) to (5), and the processes (1) to (5) may be performed by any one or more of the source device 11, the decoder 200, the destination device 12 or the decoder 300.

Process (1): a frame of image is divided into one or more parallel coding units that do not overlap each other. The parallel encoding units are independent from each other, and can be completely parallel/independently encoded and decoded, such as the parallel encoding unit 1 and the parallel encoding unit 2 shown in fig. 4.

Process (2): for each parallel coding unit, the parallel coding unit can be divided into one or more independent coding units which are not overlapped with each other, and the independent coding units can be independent of each other, but can share some parallel coding unit header information.

For example, the individual coding units are w_ lcu wide and h_ lcu high. If the parallel coding unit is divided into an independent coding unit, the size of the independent coding unit is identical to that of the parallel coding unit; otherwise, the width of the individual coding units should be larger than high (unless it is an edge region).

Typically, the individual coding units may be fixed w_ lcu ×h_ lcu, w_ lcu and h_ lcu are all to the power N of 2 (N. Gtoreq.0), e.g. the individual coding units have the following dimensions: 128×4, 64×4, 32×4, 16×4,8×4, 32×2, 16×2, or 8×2, etc.

As one possible example, the independent coding unit may be a fixed 128×4. If the parallel coding units are 256×8 in size, the parallel coding units can be equally divided into 4 independent coding units; if the parallel coding unit has a size of 288×10, the parallel coding unit is divided into: the first/second behavior is 2 independent coding units of 128×4+1 32×4; the third behavior is 2 128×2+1 32×2 independent coding units.

It should be noted that the independent encoding unit may include three components of luminance Y, chrominance Cb, and chrominance Cr, or three components of red (red, R), green (G), and blue (blue, B), or may include only one of them. If the independent coding unit comprises three components, the sizes of the three components may be identical or different, and are specifically related to the input format of the image.

Process (3): for each independent coding unit, the independent coding units can be divided into one or more coding units which are not overlapped with each other, and each coding unit in the independent coding units can be mutually dependent, for example, a plurality of coding units can perform mutual reference pre-decoding.

If the coding units are the same size as the individual coding units (i.e. the individual coding units are divided into only one coding unit), then the size may be all the sizes described in process (2).

If an independent coding unit is divided into a plurality of coding units that do not overlap each other, examples of possible divisions are: horizontal bisection (the coding units have the same height as the individual coding units but different widths may be 1/2,1/4,1/8,1/16, etc.), vertical bisection (the coding units have the same width as the individual coding units but different heights may be 1/2,1/4,1/8,1/16, etc.), horizontal and vertical bisection (quadtree partitioning), etc., preferably horizontal bisection.

The width of the coding unit is w_cu and the height is h_cu, and its width should be greater than the height (unless it is an edge region). In general, the coding units may be fixed w_cu x h_cu, w_cu and h_cu are each 2N powers (N is equal to or greater than 0), such as 16x4,8x4, 16x2,8x 1,4x1, etc.

As one possible example, the coding unit may be a fixed 16x4. If the size of the independent coding unit is 64x4, the independent coding unit can be equally divided into 4 coding units; if the size of the independent coding unit is 72x4, the coding unit is divided into: 4 16x4+1 8x4.

It should be noted that the encoding unit may include three components (or three components of red R, green G, and blue B) of luminance Y, chrominance Cb, and chrominance Cr, or may include only one component. If three components are included, the sizes of the several components may or may not be identical, particularly in relation to the image input format.

It is noted that the process (3) is an optional step in the codec method, and the codec may perform the encoding/decoding of the residual coefficient (or residual value) on the independent coding unit obtained in the process (2).

Process (4): for the coding unit, the coding unit may be further divided into one or more Prediction Groups (PGs) that do not overlap with each other, where the PGs may also be simply referred to as groups, and each PG performs encoding and decoding according to a selected Prediction mode to obtain a Prediction value of the PG, and forms a Prediction value of the whole coding unit, and obtains a residual value of the coding unit based on the Prediction value and an original value of the coding unit.

Process (5): based on the residual values of the coding units, the coding units are grouped to obtain one or more non-overlapping Residual Blocks (RBs), and the residual coefficients of each RB are encoded and decoded according to a selected mode to form a residual coefficient stream. Specifically, the residual coefficients can be classified into two types, that is, the residual coefficients are transformed and the residual coefficients are not transformed.

Wherein, the selected mode of the residual coefficient coding method in the process (5) may include, but is not limited to, any of the following: semi-fixed length coding, exponential Golomb (Golomb) coding, golomb-Rice coding, truncated unary coding, run-length coding, direct coding of the original residual value, etc.

For example, the encoder may directly encode coefficients within the RB.

For another example, the encoder may also transform the residual block, such as DCT, DST, hadamard transform, and encode the transformed coefficients.

As a possible example, when the RB is small, the encoder may directly uniformly quantize each coefficient within the RB and then binarize the encoded. If the RB is large, it may be further divided into a plurality of coefficient sets (coefficient group, CG), and each CG is uniformly quantized and binarized. In some embodiments of the present application, the Coefficient Group (CG) and the Quantization Group (QG) may be the same.

The following describes an exemplary part of the residual coefficient coding in a half-length coding manner. First, a maximum value of absolute values of residuals within one RB block is defined as a modified maximum (mm). Next, the number of coded bits of the residual coefficient in the RB block is determined (the number of coded bits of the residual coefficient in the same RB block is identical). For example, if the Critical Limit (CL) of the current RB block is 2 and the current residual coefficient is 1, 2 bits are required for encoding the residual coefficient 1, which is denoted as 01. If the CL of the current RB block is 7, the residual coefficient of 8-bit and the sign bit of 1-bit are represented. The CL is determined to find the minimum M value that satisfies the current sub-block that all residuals are within the range of [ -2 (M-1), 2 (M-1) ]. If two boundary values of-2 (M-1) and 2 (M-1) exist at the same time, M should be increased by 1, namely, all residuals of the current RB block need to be encoded by M+1 bits; if there is only one of the two boundary values of-2 (M-1) and 2 (M-1), then a tracking bit needs to be encoded to determine whether the boundary value is-2 (M-1) or 2 (M-1); if none of the residuals exist either-2 (M-1) and 2 (M-1), then the tracking bit need not be encoded.

In addition, for some special cases, the encoder may also directly encode the original values of the image instead of the residual values.

It should be noted that, the prediction unit in the embodiment of the present application may be a coding unit obtained by dividing in the above process (3), or may be a prediction group obtained by dividing the coding unit.

The image encoding method and the image decoding method according to the present application will be described in detail below with reference to the above-described frame diagrams of the codec system shown in fig. 1, the encoder shown in fig. 2, and the decoder shown in fig. 3.

First, an embodiment of the present application provides an image encoding method, which is applied to an encoding apparatus, and as shown in fig. 5, may include S401 to S405.

S401, predicting a first part of the current prediction unit by adopting a first prediction mode, and predicting a second part of the current prediction unit by adopting a second prediction mode to obtain a predicted value of the current prediction unit.

It should be appreciated that the prediction modes of the current prediction unit include at least a first prediction mode for predicting a first portion of the current prediction unit and a second prediction mode for predicting a second portion of the current prediction unit, and the first prediction mode is different from the second prediction mode.

The second portion is a portion of the current prediction unit that does not overlap the first portion, i.e., the first portion and the second portion do not overlap.

In one example, the second portion may be a portion of the current prediction unit other than the first portion, i.e., the current prediction unit is composed of the first portion and the second portion.

In another example, the current prediction unit may further include at least one other portion other than the first portion and the second portion described above, for example, the current prediction unit may further include a third portion, wherein the first portion, the second portion, and the third portion do not overlap each other. In one example, the current prediction unit is composed of a first portion, a second portion, and a third portion.

In a possible implementation, the pixels in the first part of the current prediction unit are predicted based on reconstructed pixels outside the current prediction unit, i.e. the prediction process of predicting a certain pixel in the first part does not have to depend on reconstructed pixels within the current prediction unit (including the first part), i.e. all pixels in the first part of the current prediction unit are independent of each other. In this case all pixels in the first part may be predicted in parallel using a first prediction mode, i.e. the pixels of the first part are predicted as a group, which parallel prediction mode corresponds to the first prediction mode, which may also be referred to as group prediction mode.

In one example, when pixels in the first portion are predicted in parallel based on the first prediction mode, the method for predicting each pixel in the first portion may be the same or different, for example, all pixels in the first portion may be predicted by using an angle prediction method, or a part of pixels in the first portion may be predicted by using an angle prediction method, and another part of pixels may be predicted by using a Direct Current (DC) prediction method or a bilinear interpolation (Bilinear Interpolation) prediction method.

Further, in one possible implementation, the pixels in the second portion of the current prediction unit are predicted based on reconstructed pixels outside of the current prediction unit and reconstructed pixels within the current prediction unit, or the pixels in the second portion of the current prediction unit are predicted based on reconstructed pixels within the current prediction unit. That is, when a certain pixel in the second portion is predicted, it is necessary to rely on a reconstructed pixel outside the current prediction unit and a reconstructed pixel within the current prediction unit, or when a certain pixel in the second portion is predicted, it is necessary to rely on a reconstructed pixel within the current prediction unit, that is, the prediction process of all pixels in the second portion of the current prediction unit may depend on each other. In this case the pixels in the second part may be predicted pixel by pixel using a second prediction mode, which pixel by pixel prediction mode corresponds to the second prediction mode and may also be referred to as point prediction mode.

In one example, when pixels in the second portion are predicted in parallel based on the second prediction mode, the method of predicting each pixel in the second portion may be the same or different, for example, all pixels in the first portion may be predicted according to pixels above and left, or some pixels in the second portion may be predicted according to pixels above and left, and another portion may be predicted according to pixels below and left.

It should be noted that, in the above embodiment, the pixels in the second portion are predicted based on the reconstructed pixels outside the current prediction unit and the reconstructed pixels within the current prediction unit, or the pixels in the second portion are predicted based on the reconstructed pixels within the current prediction unit, where the reconstructed pixels within the current prediction unit (for predicting the pixels of the second portion) include the reconstructed pixels in the first portion, that is, the reconstructed pixels in the first portion need to be relied on when predicting the pixels in the second portion.

It will be appreciated that in the image encoding process, an image to be encoded is generally divided into a plurality of image blocks, the image blocks are predicted one by one, and then the image blocks are encoded to obtain a code stream, where the image blocks in the image to be encoded may also be referred to as encoding blocks (i.e. encoding units).

In an example, in the embodiment of the present application, the current prediction unit may be a coding block in the image to be coded, and the current prediction unit may also be one prediction unit in a plurality of prediction units (may also be referred to as a prediction group) obtained by dividing a coding block in the image to be coded, which is specifically determined according to an actual situation, and the embodiment of the present application is not limited.

If the current prediction unit is one prediction unit of a plurality of prediction units obtained by dividing a coding block, the image coding method provided in the embodiment of the present application further includes S400 before S401.

S400, dividing the current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block.

The current prediction unit is any one of a plurality of prediction units.

In this embodiment of the present application, the preset dividing manner may be any one or a combination of multiple existing dividing methods, and specifically selected according to actual requirements.

For example, referring to fig. 6, the above-mentioned preset division manner may be a vertical division manner shown in (a) of fig. 6, where the left side of (a) of fig. 6 illustrates a current coding block, and the right side illustrates a plurality of prediction units obtained by dividing the current coding block. The vertical division shown in (a) of fig. 6 is a vertically asymmetric division manner in which the number of pixels between the respective prediction units of the current coding block division is satisfied with an index of 2.

The preset division manner may be a manner of grouping strings shown in (b) of fig. 6, and similarly, the left side of (b) of fig. 6 illustrates a current coding block, and the right side illustrates a plurality of prediction units obtained by dividing the current coding block.

The above-mentioned preset dividing manner may be a combination manner of a vertical dividing manner and a string grouping manner shown in (c) of fig. 6, and similarly, the left side of (c) of fig. 6 illustrates a current coding block, and the right side illustrates a plurality of prediction units obtained by dividing the current coding block.

In one example, the preset division manner may further include a combination of a horizontal division manner and other division manners, and the related description of the division manners may refer to the prior art.

S402, determining a residual error value between the predicted value and the true value of the current predicted unit.

In this embodiment of the present application, the residual value between the predicted value and the true value of the current prediction unit may be a difference between the predicted value of the current prediction unit and the true value of the current prediction unit from pixel to pixel.

S403, carrying out transformation processing on residual values between the predicted value and the true value of the current predicted unit according to a preset transformation direction to obtain transformed residual coefficients of the current predicted unit; horizontal conversion is performed in parallel when prediction is performed in the vertical direction, and vertical conversion is performed in parallel when prediction is performed in the horizontal direction.

That is, the preset transform direction is perpendicular to the prediction direction of the current prediction unit. In one example, the current predicted direction may be understood as a predicted angle, e.g. the predicted direction may be a horizontal direction, a vertical direction or a direction at any angle to the reference direction.

In one example, when the preset transform direction is horizontal transform, the height of the current prediction unit is 1, that is, for an image block, after a row of pixels is predicted, transform processing is performed on residual values corresponding to the row of pixels in parallel; when the preset transformation direction is vertical transformation, the width of the current prediction unit is 1, that is, after a column of pixels is predicted for an image block, transformation processing is performed on residual values corresponding to the column of pixels in parallel.

In the embodiment of the present application, pixels along a prediction direction are interdependent in a process of predicting the pixels, so that when a predicted value along the prediction direction is transformed, pixels along the prediction direction are interdependent, that is, a plurality of pixels cannot be transformed in parallel along the prediction direction. Since the pixels are mutually dependent in the direction perpendicular to the prediction direction, a plurality of pixels can be converted in parallel in the direction perpendicular to the prediction direction, and thus the coding efficiency can be improved.

S404, carrying out quantization processing on the transformation residual coefficient of the current prediction unit to obtain the quantization residual coefficient of the current prediction unit.

S405, the quantized residual coefficient of the current prediction unit is encoded.

In one example, in the embodiment of the present application, the encoding device further encodes some encoding parameters of the current prediction unit, which may also be referred to as syntax elements, and the encoding parameters may include, but are not limited to, prediction mode information of the current prediction unit, for example, information indicating the above-mentioned first prediction mode and second prediction mode, and so on.

In the embodiment of the present application, after S402, S403 may not be executed, that is, the residual value may not be transformed and the residual value may be directly quantized and encoded.

The related description about the details of S404-S405 may refer to the existing quantization processing method and encoding method, and the embodiments of the present application will not be described in detail.

So far, the encoding device completes the encoding of one prediction unit, and the code stream of the prediction unit is obtained. The encoding device may then transfer the code stream to a decoding device or store the code stream.

In one example, after S402, the image encoding method provided in the present application further includes S406 to S408.

S406, performing inverse quantization processing on the quantized residual coefficient of the current prediction unit to obtain an inverse quantization coefficient of the current prediction unit.

It should be understood that the process of the inverse quantization process is the inverse of the quantization process described above.

S407, performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the predicted value and the true value of the current prediction unit; the horizontal inverse transforms are performed in parallel when predicting in the vertical direction, and the vertical inverse transforms are performed in parallel when predicting in the horizontal direction.

In one example, when the preset inverse transform direction is the horizontal inverse transform, the height of the current prediction unit is 1; when the preset inverse transform direction is the vertical inverse transform, the width of the current prediction unit is 1.

The process of the inverse transform process is the inverse process of the transform process, the inverse transform direction of the inverse transform process is perpendicular to the prediction direction of the current prediction unit, the inverse transform direction is the same as the transform direction, and the detailed description of S403 in the above embodiment may be referred to for the relevant description of the inverse transform direction, which is not repeated here.

S408, determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

In one example, the sum of the prediction value of the current prediction unit and the residual value of the current prediction unit is the reconstructed value of the current prediction unit.

In one example, after obtaining the reconstructed value of the current prediction unit, loop filtering may be further performed on the reconstructed value to obtain the reconstructed unit of the current prediction unit.

In this embodiment, after the encoding device executes S406-S408 to obtain the reconstructed value or the reconstructed unit of the current prediction unit, the reconstructed value or the reconstructed unit is cached in the buffer, and subsequently, the next prediction unit may be predicted according to the reconstructed value or the reconstructed unit of the current prediction unit.

Similarly, in the embodiment of the present application, S407 may not be performed after S406, that is, the inverse quantization coefficient may be used as the residual value without transforming the inverse quantization coefficient.

In the image coding method provided by the embodiment of the application, the coding device predicts the first part of the current unit to be predicted by adopting the first prediction mode, and the second part of the current unit to be predicted by adopting the second prediction mode.

With respect to the image encoding method shown in fig. 5, after the encoding device finishes image encoding, the code stream is transmitted to the decoding device, and accordingly, the embodiment of the present application provides an image decoding method applied to the decoding device, and the image decoding method includes S601-S603 as shown in fig. 7.

S601, analyzing a code stream, and acquiring a prediction mode of a current prediction unit and a residual value between a true value and a predicted value of the current prediction unit, wherein the prediction mode of the current prediction unit at least comprises a first prediction mode and a second prediction mode.

The first prediction mode is used for predicting a first part of the current prediction unit, the second prediction mode is used for predicting a second part of the current prediction unit, and the first prediction mode is different from the second prediction mode.

In this embodiment of the present application, after receiving an image code stream to be encoded from an encoding apparatus, a decoding apparatus parses a decoding parameter from the code stream, for example, including the prediction mode of the current prediction unit described above, and parses the code stream to obtain a residual error between a true value and a predicted value of the current prediction unit.

The second portion is a portion of the current prediction unit that does not overlap the first portion, i.e., the first portion and the second portion do not overlap. In one example, the second portion may be a portion of the current prediction unit other than the first portion, i.e., the current prediction unit is composed of the first portion and the second portion.

It should be noted that, when a pixel in the first portion of the current prediction unit is predicted based on the reconstructed pixels outside the current prediction unit, that is, when a certain pixel in the first portion is predicted, it is not necessary to rely on the reconstructed pixels in the current prediction unit (including the first portion), that is, the prediction process of all pixels in the first portion of the current prediction unit is independent of each other. In this case all pixels in the first part may be predicted in parallel using the first prediction mode, i.e. the pixels of the first part are predicted as a group, which prediction mode may be referred to as group prediction.

Further, it should be noted that the pixels in the second portion of the current prediction unit are predicted based on the reconstructed pixels outside the current prediction unit and the reconstructed pixels within the current prediction unit, or the pixels in the second portion of the current prediction unit are predicted based on the reconstructed pixels within the current prediction unit. That is, when a certain pixel in the second portion is predicted, it is necessary to rely on a reconstructed pixel outside the current prediction unit and a reconstructed pixel within the current prediction unit, or when a certain pixel in the second portion is predicted, it is necessary to rely on a reconstructed pixel within the current prediction unit, that is, the prediction process of all pixels in the second portion of the current prediction unit may depend on each other. In which case pixels in the second part may be predicted pixel by pixel using the second prediction mode, which may be referred to as point prediction.

In one example, the pixels in the second part described in the above embodiment are predicted based on reconstructed pixels outside the current prediction unit and reconstructed pixels within the current prediction unit, or the pixels in the second part are predicted based on reconstructed pixels within the current prediction unit, wherein the reconstructed pixel prediction within the current prediction unit (for predicting the pixels of the second part) comprises reconstructed pixels in the first part described above, i.e. the pixels in the second part need to be predicted dependent on reconstructed pixels in the first part of the current prediction unit.

S602, predicting a first part of the current prediction unit by adopting a first prediction mode, and predicting a second part of the current prediction unit by adopting a second prediction mode to obtain a predicted value of the current prediction unit.

In this embodiment, the decoding apparatus predicts the pixel value of the first portion of the current prediction unit from the decoded (reconstructed) pixel based on the first prediction mode, and similarly, predicts the pixel of the second portion of the current prediction unit from the decoded pixel based on the second prediction mode.

It can be appreciated that the method for predicting the current prediction unit in the image decoding process is similar to the method for predicting the current prediction unit in the image encoding process, and therefore, the description of other details of S602 may refer to the description of S401 in the above embodiment, which is not repeated here.

S603, determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

The sum of the predicted value of the current prediction unit and the residual value of the current prediction unit is the reconstructed value of the current prediction unit.

In this embodiment of the present application, after obtaining a reconstructed value or a reconstructed unit of a current prediction unit, a decoding device caches the reconstructed value or the reconstructed unit in a buffer, and subsequently, may predict a next prediction unit according to the reconstructed value or the reconstructed unit of the current prediction unit.

In this embodiment of the present application, according to the method of S601 to S603 described above, all prediction units in the image to be decoded are reconstructed, so as to obtain a reconstructed image of the image to be decoded, and the reconstructed image may be displayed on a decoding device.

In one example, referring to fig. 7, as shown in fig. 8, the above-mentioned analysis of the code stream in S601 obtains a residual value between the true value and the predicted value of the current prediction unit, specifically including S6011-S6013.

And S6011, decoding the code stream of the current prediction unit to obtain the quantization coefficient of the current prediction unit.

And S6012, performing inverse quantization processing on the quantized coefficient of the current prediction unit to obtain the inverse quantized coefficient of the current prediction unit.

S6013, performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the true value and the predicted value of the current prediction unit; the horizontal inverse transforms are performed in parallel when predicting in the vertical direction, and the vertical inverse transforms are performed in parallel when predicting in the horizontal direction.

That is, the preset inverse transform direction is perpendicular to the prediction direction of the current prediction unit. In one example, the current predicted direction may be understood as a predicted angle, e.g. the predicted direction may be a horizontal direction, a vertical direction or a direction at any angle to the reference direction.

In the embodiment of the present application, pixels along a prediction direction are interdependent in a process of predicting the pixels, so that when a predicted value along the prediction direction is transformed, pixels along the prediction direction are interdependent, that is, a plurality of pixels cannot be transformed in parallel along the prediction direction. And each pixel is mutually dependent along the direction perpendicular to the prediction direction, so that a plurality of pixels can be subjected to conversion processing in parallel along the direction perpendicular to the prediction direction, and the efficiency of image reconstruction can be improved.

In this embodiment, after the decoding device parses the quantized coefficient of the current prediction unit, the process of performing inverse quantization processing and inverse transformation processing on the quantized coefficient is similar to the processes of S406-S407 described above, so the description of S6012-S6013 may refer to the related description of S406-S407 in the above embodiment, which is not repeated here.

In the embodiment of the present application, after S6012, S6013 may not be executed, that is, the inverse quantization coefficient may be used as the residual value without being transformed.

It will be appreciated that in the image decoding process, the image to be decoded is typically divided into a plurality of image blocks, and the image blocks are predicted one by one, and the image blocks in the image to be decoded may also be referred to as encoding blocks.

In an example, in the embodiment of the present application, the current prediction unit may be a coded block in the image to be decoded, and the current prediction unit may also be one prediction unit in a plurality of prediction units obtained by dividing a coded block in the image to be decoded, which is specifically determined according to an actual situation, and the embodiment of the present application is not limited.

As shown in fig. 8, if the current prediction unit is one prediction unit of a plurality of prediction units obtained by dividing one coding block, the image decoding method provided in the embodiment of the present application further includes S600 before S601.

S600, dividing the current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block.

The current prediction unit is any one of a plurality of prediction units.

In this embodiment of the present application, the preset dividing manner may be any one or a combination of multiple existing dividing methods, for example, may include a vertical dividing manner, a horizontal dividing manner, a serial grouping manner, and the like, which are specifically selected according to actual needs, and the embodiment of the present application is not limited.

Further description of the partitioning in S600 may be detailed description of S400 in the above embodiments, which is not repeated here.

In summary, in the image decoding method provided by the embodiment of the present application, the decoding device predicts the first portion of the current unit to be predicted by using the first prediction mode, and predicts the second portion of the current unit to be predicted by using the second prediction mode.

For example, referring to fig. 9, assuming that the size of the current prediction unit is 8×4 (8 refers to the number of pixels in the image height direction, 4 refers to the number of pixels in the image width direction), in the above-described embodiment of image encoding and image decoding, in predicting the current prediction unit, a first prediction mode may be used for the pixels in a first portion of the current prediction unit, for example, the pixels of the previous row and/or the pixels of the previous column of the current prediction unit may be used as reference pixels for the pixels in the first portion, the pixels in the first portion may be predicted, and all the pixels in the first portion may be predicted in parallel. For example, in the first prediction mode, at least one of an angle prediction method, a Direct Current (DC) prediction method, and a bilinear interpolation (Bilinear Interpolation) prediction method may be used to predict the pixels of the first portion. For the pixels in the second portion of the current prediction unit, prediction may be performed by predicting one pixel one by one, for example, a pixel value of the pixel may be predicted according to other pixels near the pixel, for example, in the second prediction mode, at least one of an adaptive median prediction (median adaptive prediction, MAP) method, an improved MAP (i.e., MMAP), and a mid-point prediction (MPP) method may be used to predict the pixels in the second portion.

As shown in fig. 10, taking the current prediction unit of 8×4 as an example, the pixels marked with 1,2,3, …,16 in fig. 10 are predicted by the second prediction mode, for example, the weighted average of the reconstructed pixels above, below, left and right in fig. 10 may be used as the prediction value when predicting the pixel 1, pixel 2, pixel 3, pixel 5, pixel 6, pixel 7, pixel 8, pixel 9, pixel 10 and pixel 11 in fig. 10 according to the weighted average of the reconstructed pixels above, below, left and right in the pixel to be predicted.

It should be noted that, since some pixels located at the edge of the current prediction unit in the second portion may not have reference pixels in at least one of the directions above, below, left or right, a special prediction mode is required to predict the pixels in this case. For example, pixel 4 and pixel 12 in fig. 10 do not have a reference pixel to the right, and therefore the pixel value of that pixel can be predicted from its pixels to the left, above, and below; for another example, the

pixels

13, 14, 15, and 16 in fig. 10 do not have the reference pixel below, and therefore, the pixel value of the pixel can be predicted from the pixels to the left, to the right, and above.

Referring to fig. 10, it can be seen that a part of pixels in the second portion are predicted based on the reconstructed pixels outside the current prediction unit and the reconstructed pixels within the current prediction unit, for example, pixel 1, pixel 2, pixel 3, pixel 4, pixel 5, and pixel 13 are predicted based on the reconstructed pixels outside the current prediction unit and the reconstructed pixels within the current prediction unit; some of the pixels in the second portion are predicted based on reconstructed pixels within the current prediction unit, e.g., pixel 6, pixel 7, pixel 8, pixel 9, pixel 10, pixel 11, pixel 12, pixel 14, pixel 15, and pixel 16 are predicted from reconstructed pixels within the current prediction unit.

The above-described prediction process for the current prediction unit is intra prediction.

For example, referring to fig. 11, when the residual value of the current prediction unit is transformed during the above-mentioned image encoding process, when the pixels of the first line of the current prediction unit need to be predicted according to the pixels of the previous line, that is, the prediction direction is the vertical direction, then transforming the pixel value of one prediction in the vertical direction depends on the pixels of the previous line, and transforming the pixels in the direction perpendicular to the prediction direction (that is, the horizontal direction) is independent of each other, in which case, the predicted pixels may be transformed in the horizontal direction (the transforming process shown in fig. 9 is a one-dimensional transforming process).

The embodiment of the application also provides an image encoding method, which is applied to an encoding device, and as shown in fig. 12, the image encoding method may include S1101-S1105.

S1101, predicting the current prediction unit by adopting an angle prediction mode to obtain a prediction value of the current prediction unit. The angle prediction mode satisfies a preset condition that the number of pixels |m| of the predicted pixel point and the current pixel point, which are determined according to the prediction angle, are spaced in the image height direction, is an integer, the number of pixels |n| of the predicted pixel point and the current pixel point, which are determined according to the prediction angle, are spaced in the image width direction, is an integer, and |m| or |n| is the minimum value among the desirable integer values.

Referring to fig. 13, that is, the above-described angle prediction mode satisfies

M and N are integers; where θ represents a prediction angle, |m| represents the number of pixels at an interval between a current pixel and a predicted pixel of the current pixel in the image height direction, |n| represents the number of pixels at an interval between a current pixel and a predicted pixel of the current pixel in the image width direction, and |m| is satisfied->

The minimum value or |N| of all absolute values of the values is satisfied

The minimum of all absolute values of the values, || represents the absolute value.

Further, in one example, M and N above also satisfy

n is an integer.

In the prediction process of the current prediction unit, the original complex multiplication and division operation is changed into simple shift operation, so that the complexity of calculation can be reduced, for example, multiplication by 8 can be equivalently shifted left by 3 units, namely<<3。

If the current prediction unit is one prediction unit of a plurality of prediction units obtained by dividing one coding block, the image coding method provided in the embodiment of the present application further includes S1100 before S1100.

S1100, dividing the current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block.

The current prediction unit is any one of a plurality of prediction units.

It should be noted that, the dividing manner in S1100 is similar to the dividing manner in S400 in the above embodiment, and further description of S1100 may be detailed description of S400 in the above embodiment, which is not repeated here.

S1102, determining a residual error value between a predicted value and a true value of the current prediction unit.

S1103, carrying out transformation processing on residual values between predicted values and true values of the current predicted unit according to a preset transformation direction to obtain transformed residual coefficients of the current predicted unit; horizontal conversion is performed in parallel when prediction is performed in the vertical direction, and vertical conversion is performed in parallel when prediction is performed in the horizontal direction.

That is, the preset transform direction is perpendicular to the prediction direction of the current prediction unit. In one example, the current predicted direction may be understood as a predicted angle, e.g. the predicted direction may be a horizontal direction, a vertical direction or a direction at any angle to the reference direction. For the detailed description of the transformation direction, reference may be made to the description of the transformation direction in S403, which is not repeated here.

In one example, when the preset transform direction is the horizontal inverse transform, the height of the current prediction unit is 1; when the preset transform direction is the vertical inverse transform, the width of the current prediction unit is 1.

S1104, carrying out quantization processing on the transformation residual coefficient of the current prediction unit to obtain the quantization residual coefficient of the current prediction unit.

S1105, the quantized residual coefficient of the current prediction unit is encoded.

In one example, in the embodiment of the present application, the encoding device further encodes some encoding parameters of the current prediction unit, which may also be referred to as syntax elements, and the encoding parameters may include, but are not limited to, prediction mode information of the current prediction unit, for example, information indicating the above-mentioned angular prediction mode, and the like.

In the embodiment of the present application, after S1102, S1103 may not be executed, that is, the residual value may not be transformed, and the residual value may be directly quantized and recoded.

In one example, after S1104 above, the image encoding method provided in the present application further includes S1106-S1108.

S1106, performing inverse quantization processing on the quantized residual coefficient of the current prediction unit to obtain an inverse quantization coefficient of the current prediction unit.

S1107, performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction, so as to obtain a residual error value between the predicted value and the true value of the current prediction unit, wherein the preset inverse transformation direction is perpendicular to the predicted direction of the current prediction unit.

The process of the inverse transform process is an inverse process of the above transform process, the inverse transform direction of the inverse transform process is perpendicular to the prediction direction of the current prediction unit, the inverse transform direction is the same as the above transform direction, and the detailed description of S1103 in the above embodiment may be referred to for the relevant description of the inverse transform direction, which is not repeated here.

S1108, determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

In this embodiment, after the encoding device executes S1106-S1108 to obtain the reconstructed value or the reconstructed unit of the current prediction unit, the reconstructed value or the reconstructed unit is cached in the buffer, and subsequently, the next prediction unit may be predicted according to the reconstructed value or the reconstructed unit of the current prediction unit.

Similarly, in the embodiment of the present application, after S1106, S1107 may not be performed, that is, the inverse quantization coefficient may be used as the residual value without transforming the inverse quantization coefficient.

According to the image coding method provided by the embodiment of the application, the coding device can adopt the angle prediction mode to predict the current prediction unit, and the angle prediction mode meets the following conditions: the number of pixels, which are determined according to the prediction angle, of the pixels, which are spaced between the prediction pixel point and the current pixel point in the image height direction, is an integer, and the number of pixels, which are spaced between the prediction pixel point and the current pixel point in the image width direction, determined according to the prediction angle, is an integer, namely, the reconstructed pixel value can be directly used as the prediction value of the pixel to be predicted, so that pixel interpolation operation can be avoided, the calculated amount can be reduced, the image coding efficiency can be improved, and the image coding performance can be improved.

With respect to the image encoding method shown in fig. 12, after the encoding device completes image encoding, the code stream is transmitted to the decoding device, and accordingly, the embodiment of the present application provides an image decoding method applied to the decoding device, and as shown in fig. 14, the image decoding method includes S1301-S1303.

S1301, analyzing a code stream, and acquiring a prediction mode of a current prediction unit and a residual value between a true value and a predicted value of the current prediction unit, wherein the prediction mode of the current prediction unit is an angle prediction mode, the angle prediction mode meets a preset condition, the preset condition is that the number M of pixels between a predicted pixel point and the current pixel point, which are determined according to a prediction angle, in the image height direction is an integer, the number N of pixels between the predicted pixel point and the current pixel point, which are determined according to the prediction angle, in the image width direction is an integer, and M or N is the minimum value in the desirable integer values.

That is, the above-described angle prediction mode satisfies

The minimum value or |N| of all absolute values of the values is satisfied +.>

Further, in one example, M and N above also satisfy

n is an integer.

Compared with the prior art, when the current prediction unit is predicted, the pixel number of the interval between the current pixel point and the predicted pixel point of the current pixel point is an integer in the image height direction, and the pixel number of the interval between the current pixel point and the predicted pixel point of the current pixel point is also an integer in the image width direction, so that the reconstructed pixel value can be directly used as the predicted value of the pixel to be predicted, pixel interpolation operation can be avoided, the calculated amount can be reduced, and the image coding efficiency is improved.

For a more detailed description of the angle prediction mode, reference may be made to the description of S1101, which is not repeated here.

S1302, predicting the current prediction unit by adopting a prediction mode of the current prediction unit to obtain a prediction value of the current prediction unit.

It should be noted that, the prediction method of the current image in the image decoding process corresponds to, i.e., is similar to, the prediction method of the image in the image encoding process. Therefore, the detailed description of S1302 may refer to the detailed description of S1101 described above, and will not be repeated here.

S1303, determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

In the embodiment of the present application, according to the method of S1301-S1303 described above, all prediction units in the image to be decoded are reconstructed, so as to obtain a reconstructed image of the image to be decoded, and the reconstructed image may be displayed on a decoding device.

In one example, the above-mentioned parsing code stream in S1301 obtains a residual value between the true value and the predicted value of the current prediction unit, specifically including S13011-S13011.

S13011, decoding the code stream of the current prediction unit to obtain the quantization coefficient of the current prediction unit.

S13012, performing inverse quantization processing on the quantized coefficient of the current prediction unit to obtain the inverse quantized coefficient of the current prediction unit.

S13013, performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the true value and the predicted value of the current prediction unit; the horizontal inverse transforms are performed in parallel when predicting in the vertical direction, and the vertical inverse transforms are performed in parallel when predicting in the horizontal direction.

In this embodiment, after the decoding device parses the quantized coefficient of the current prediction unit, the process of performing inverse quantization processing and inverse transformation processing on the quantized coefficient is similar to the above-mentioned S1106-S1107, so the description of S13011-S13013 may refer to the related description of S1106-S1107 in the above-mentioned embodiment, which is not repeated here.

In the embodiment of the present application, S13013 may not be executed after S13012, that is, the inverse quantization coefficient may be used as the residual value without being transformed.

Referring to fig. 14, as shown in fig. 15, if the current prediction unit is one prediction unit of a plurality of prediction units obtained by dividing one coding block, the image decoding method provided in the embodiment of the present application further includes S1300 before S1301.

S1300, dividing the current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block.

The current prediction unit is any one of a plurality of prediction units.

Further description of the division manner in S1300 may be detailed description of S1100 in the above embodiment, which is not repeated here.

In summary, in the image decoding method provided in the embodiment of the present application, the decoding device may predict the current prediction unit using an angle prediction mode, where the angle prediction mode satisfies: the number of pixels, which are determined according to the prediction angle, of the pixels, which are spaced between the prediction pixel point and the current pixel point in the image height direction, is an integer, and the number of pixels, which are spaced between the prediction pixel point and the current pixel point in the image width direction, determined according to the prediction angle, is an integer, namely, the reconstructed pixel value can be directly used as the prediction value of the pixel to be predicted, so that pixel interpolation operation can be avoided, the calculated amount can be reduced, the image coding efficiency can be improved, and the image decoding performance can be improved.

For example, taking an image block with a current prediction unit of 8×4 as an example, as shown in fig. 16, in the embodiment of image encoding and image decoding described above, when the current prediction unit is predicted using the angle prediction mode, the prediction directions of all pixels in each row of the current prediction unit are the same and the motion vectors are also the same, and the motion vectors of different rows are different, and the motion vectors can be understood as horizontal offsets from the current position to the predicted position. For example, referring to fig. 16, the prediction directions of all pixels of the 1 st line of the current prediction unit are the same and the motion vectors are the same, but the prediction directions of all pixels of the 1 st line are the same as the prediction directions of all pixels of the 2 nd line, but the motion vectors are different.

In connection with the current prediction unit shown in fig. 16, after determining the prediction angle (i.e., the prediction direction), all pixels in the current prediction unit are predicted according to the above-described angle prediction mode, assuming that the 1 st line of the current prediction unit encodes the m-th line of the image (or the encoded block), each line of the current prediction unit is predicted separately for the prediction angle shown in fig. 16, and the specific procedure is as shown in fig. 17.

As can be seen from fig. 17 (a): in the current prediction unit, the predicted values of all the pixels in the m-2 th row are predicted based on the reconstructed pixels in the m-2 th row, and the pixel P11 in the m-2 th row is taken as the predicted value of the 1 st pixel in the m-2 th row, so that the predicted values of the 8 pixels in the m-2 th row are sequentially the pixel values of the pixels P11, P12, P13, P14, P15, P16, P17 and P18 in the m-2 th row. And according to this angular prediction mode, the reconstructed pixel value is directly taken as the prediction value of the pixel in the m-th line, that is, the pixel value in the m-th line is obtained without interpolating a plurality of pixel values when the pixel in the m-th line is predicted.

As can be seen in connection with (b) in fig. 17: in the current prediction unit, the predicted values of all the pixels in the m+1th row are predicted based on the reconstructed m-1th row pixels, and the pixel P21 in the m-1th row is taken as the predicted value of the 1 st pixel in the m+1th row, so that the predicted values of the 8 pixels in the m+1th row are the pixel values of the pixels P21, P22, P23, P24, P25, P26, P27 and P28 in turn. Similarly, according to this angle prediction mode, the reconstructed pixel value is directly used as the prediction value of the m+1th row of pixels, that is, the pixel value of the m row is obtained without interpolating a plurality of pixel values when the m+1th row of pixels is predicted.

As can be seen in connection with (c) in fig. 17: in the current prediction unit, the predicted values of all pixels in the m+2 th row are predicted based on the reconstructed pixels in the m-2 th row, and the pixel P12 in the m-2 th row is taken as the predicted value of the 1 st pixel in the m+2 th row, so that the predicted values of the 8 pixels in the m+2 th row are the pixel values of the pixels P12, P13, P14, P15, P16, P17, P18 and P19 in turn. Similarly, according to this angle prediction mode, the reconstructed pixel value is directly used as the prediction value of the m+2th line pixel, that is, the pixel value of the m line is obtained without interpolating a plurality of pixel values when the m+2th line pixel is predicted.

As can be seen in connection with (d) in fig. 17: in the current prediction unit, the predicted values of all the pixels in the m+3 th row are predicted based on the reconstructed m-1 st row of pixels, and the pixel P22 in the m-1 st row is taken as the predicted value of the 1 st pixel in the m+3 th row, so that the predicted values of the 8 pixels in the m+3 th row are the pixel values of the pixels P22, P23, P24, P25, P26, P27, P28 and P29 in turn. Similarly, according to this angle prediction mode, the reconstructed pixel value is directly used as the prediction value of the m+3rd line pixel, that is, the pixel value of the m line is obtained without interpolating a plurality of pixel values when the m+3rd line pixel is predicted.

In connection with fig. 17, the prediction process between the respective lines of the current prediction unit has a certain rule, for example, the 1 st line and the 3 rd line of the current prediction unit have an offset relationship in the image width direction (i.e., the horizontal direction) with respect to the 1 st line of the motion vector, and the 1 st line and the 3 rd line of the motion vector also have an offset relationship in the image height direction (i.e., the vertical direction), for example, the 3 rd motion vector is a result of the 1 st line of the motion vector being offset by 1 pixel in the image width direction, and the 3 rd line of the motion vector being a result of the 1 st line of the motion vector being offset by 2 pixels in the image height direction.

In summary, when the current prediction unit is predicted according to the angle prediction mode provided by the embodiment of the application, the current prediction unit is predicted by adopting the pixels reconstructed in the first two rows of the current prediction unit, and compared with the scheme of predicting the pixel value of the current position according to the pixels in the first row of the current prediction position in the prior art, the embodiment of the application can avoid pixel interpolation operation when the current prediction unit is predicted, so that the calculated amount can be reduced, and the image coding efficiency is improved.

Accordingly, the embodiment of the present application provides an image decoding device, which may be any image decoding device in a call process, and according to the above method example, the image decoding device may be divided into functional modules, for example, each functional module may be divided into corresponding functions, or two or more functions may be integrated into one processing module. The integrated modules may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present invention, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

Fig. 18 shows a possible configuration diagram of the image decoding apparatus related to the above-described embodiment in the case where respective functional blocks are divided with corresponding respective functions. As shown in fig. 18, the image decoding apparatus includes an analysis module 1701, a prediction module 1702, and a reconstruction module 1703.

In one embodiment, the parsing module 1701 is configured to parse a code stream to obtain a prediction mode of a current prediction unit and a residual value between a true value and a predicted value of the current prediction unit; the prediction modes of the current prediction unit at least include a first prediction mode for predicting a first portion of the current prediction unit and a second prediction mode for predicting a second portion of the current prediction unit, where the first prediction mode is different from the second prediction mode, for example, S601 in the above-described method embodiment is performed. The prediction module 1702 is configured to predict a first portion of the current prediction unit using a first prediction mode, and predict a second portion of the current prediction unit using a second prediction mode, so as to obtain a predicted value of the current prediction unit, for example, execute S602 in the above method embodiment. A reconstruction module 1703, configured to determine a reconstruction value S603 of the current prediction unit according to the prediction value of the current prediction unit and the residual value of the current prediction unit.

In one example, the image decoding apparatus provided in the embodiment of the present application further includes an inverse quantization module 1704 and an inverse transform module 1705, where the parsing module 1701 is specifically configured to decode a code stream of a current prediction unit to obtain a quantization coefficient of the current prediction unit, for example, S6011 in the embodiment is executed; the inverse quantization module 1704 is configured to perform inverse quantization processing on the quantization coefficient of the current prediction unit, so as to obtain an inverse quantization coefficient of the current prediction unit; for example, S6012 in the above embodiment is performed; the inverse transform module 1705 is configured to perform inverse transform processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transform direction, so as to obtain a residual value between the predicted value and the true value of the current prediction unit, where the preset inverse transform direction is perpendicular to the predicted direction of the current prediction unit, for example, S6013 in the above embodiment is performed.

In an example, the image decoding apparatus provided in the embodiment of the present application further includes a grouping module, where the grouping module is configured to divide the current coding block according to a preset division manner to obtain a plurality of prediction units of the current coding block, where the current prediction unit is any one of the plurality of prediction units, for example, S600 in the embodiment described above is performed.

In another embodiment, the parsing module 1701 is configured to parse a code stream, obtain a prediction mode of a current prediction unit and a residual value between a prediction value and a true value of the current prediction unit, where the prediction mode of the current prediction unit is an angle prediction mode, and the angle prediction mode satisfies a preset condition, where the preset condition is that a number |m| of pixels, which are determined according to a prediction angle, between a predicted pixel point and the current pixel point in an image height direction is an integer, a number |n| of pixels, which are determined according to the prediction angle, between the predicted pixel point and the current pixel point in the image width direction is an integer, and |m| or |n| is a minimum value of the desirable integer values, where the |m| represents taking an absolute value, for example, 1301 in the embodiment is executed; the prediction module 1702 is configured to predict a current prediction unit by using a prediction mode of the current prediction unit to obtain a predicted value of the current prediction unit, for example, execute 1302 in the above embodiment; the reconstruction module 1703 is configured to determine a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual value of the current prediction unit, for example, to perform 1303 in the above embodiment.

The parsing module 1701 is specifically configured to decode a code stream of the current prediction unit to obtain a quantization coefficient of the current prediction unit, for example, execute S13011 in the above embodiment; the inverse quantization module 1704 is configured to perform inverse quantization processing on the quantization coefficient of the current prediction unit, so as to obtain an inverse quantization coefficient of the current prediction unit; for example, S13012 in the above-described embodiment is executed; the inverse transform module 1705 is configured to perform inverse transform processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transform direction, so as to obtain a residual value between the predicted value and the true value of the current prediction unit, where the preset inverse transform direction is perpendicular to the predicted direction of the current prediction unit, for example, S13013 in the above embodiment is performed.

In one example, the grouping module is configured to divide the current coding block according to a preset division manner to obtain a plurality of prediction units of the current coding block, where the current prediction unit is any one of the plurality of prediction units, for example, S1200 in the foregoing embodiment is performed.

The modules of the image decoding apparatus may also be used to perform other actions in the above method embodiments, and all relevant content of each step related to the above method embodiments may be referred to the functional descriptions of the corresponding functional modules, which are not repeated herein.

In the case of using an integrated unit, fig. 19 shows another possible structural diagram of the image decoding apparatus involved in the above-described embodiment. As shown in fig. 19, the image decoding apparatus provided in the embodiment of the present application may include: a processing module 1801 and a communication module 1802. The processing module 1801 may be used to control and manage the operations of the image decoding apparatus, for example, the processing module 1801 may be used to support the image decoding apparatus to perform S601-S603, S600, S1301-S1303, S1300, etc. in the above-described method embodiments, and/or other processes for the techniques described herein. The communication module 1802 may be used to support communication of the image decoding apparatus with other network entities, such as with an image encoding apparatus. In one example, as shown in fig. 19, the image decoding apparatus may further include a storage module 1803 for storing program codes and data of the image decoding apparatus, for example, storing a decoded image.

The processing module 1801 may be a processor, for example, the processor may be the decoder 122 in fig. 1. The communication module 1802 may be a transceiver, a transceiver circuit, or a communication interface, etc., such as the communication interface 121 of fig. 1, and the storage module 1803 may be a memory.

For more details on the implementation of the above functions by the modules comprised in the above image decoding apparatus, reference is made to the description of the previous method embodiments, which is not repeated here.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be wholly or partly implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more servers, data centers, etc. that can be integrated with the available medium. The usable medium may be a magnetic medium (e.g., floppy disk, magnetic tape), an optical medium (e.g., digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., solid state disk (solid state drives, SSD)), or the like.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above. The specific working processes of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which are not described herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the various embodiments of the present application. And the aforementioned storage medium includes: flash memory, removable hard disk, read-only memory, random access memory, magnetic or optical disk, and the like.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image decoding method, comprising:

analyzing a code stream, and acquiring a prediction mode of a current prediction unit and a residual error value between a true value and a predicted value of the current prediction unit; the prediction mode of the current prediction unit at least comprises a first prediction mode and a second prediction mode, wherein the first prediction mode is used for predicting a first part of the current prediction unit, the second prediction mode is used for predicting a second part of the current prediction unit, and the first prediction mode is different from the second prediction mode;

predicting a first part of the current prediction unit by adopting the first prediction mode, and predicting a second part of the current prediction unit by adopting the second prediction mode to obtain a predicted value of the current prediction unit;

And determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

pixels in the first portion are predicted based on reconstructed pixels outside of the current prediction unit;

pixels in the second portion are predicted based on reconstructed pixels outside of the current prediction unit and reconstructed pixels within the current prediction unit, or pixels in the second portion are predicted based on reconstructed pixels within the current prediction unit.

3. The method of claim 2, wherein the step of determining the position of the substrate comprises,

pixels in the second portion are predicted based on reconstructed pixels outside of the current prediction unit and reconstructed pixels within the current prediction unit, or pixels in the second portion are predicted based on reconstructed pixels within the current prediction unit; reconstructed pixels within the current prediction unit include reconstructed pixels in the first portion.

4. A method according to any one of claims 1 to 3, wherein said parsing the code stream to obtain residual values between the true value and the predicted value of the current prediction unit comprises:

Decoding the code stream of the current prediction unit to obtain a quantization coefficient of the current prediction unit;

performing inverse quantization processing on the quantized coefficient of the current prediction unit to obtain an inverse quantized coefficient of the current prediction unit;

performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the predicted value and the true value of the current prediction unit; the horizontal inverse transforms are performed in parallel when predicting in the vertical direction, and the vertical inverse transforms are performed in parallel when predicting in the horizontal direction.

5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,

when the preset inverse transformation direction is horizontal inverse transformation, the height of the current prediction unit is 1;

and when the preset inverse transformation direction is vertical inverse transformation, the width of the current prediction unit is 1.

6. The method according to any one of claims 1 to 5, further comprising:

dividing a current coding block according to a preset dividing mode to obtain a plurality of prediction units of the current coding block, wherein the current prediction unit is any one of the plurality of prediction units.

7. An image decoding method, comprising:

analyzing a code stream, and acquiring a prediction mode of a current prediction unit and a residual error value between a prediction value and a true value of the current prediction unit;

the prediction mode of the current prediction unit is an angle prediction mode, the angle prediction mode meets a preset condition, the preset condition is that the number |M| of pixels, which are determined according to the prediction angle, of the predicted pixel points and the current pixel points and are spaced in the image height direction is an integer, the number |N| of pixels, which are determined according to the prediction angle and are spaced in the image width direction, of the predicted pixel points and the current pixel points is an integer, and |M| or |N| is the minimum value in the acceptable integer values, and || represents an absolute value;

predicting the current prediction unit by adopting a prediction mode of the current prediction unit to obtain a prediction value of the current prediction unit;

8. The method of claim 7, wherein the step of determining the position of the probe is performed,

n is an integer.

9. The method according to claim 7 or 8, wherein said parsing the bitstream to obtain a residual value between the predicted value and the true value of the current prediction unit comprises:

performing inverse transformation processing on the inverse quantization coefficient of the current prediction unit according to a preset inverse transformation direction to obtain a residual error value between the predicted value and the true value of the current prediction unit; when prediction is performed in the vertical direction, horizontal inverse transformation is performed in parallel; when predicting in the horizontal direction, the inverse vertical transform is performed in parallel.

10. The method of claim 9, wherein the step of determining the position of the substrate comprises,

11. The method according to any one of claims 7 to 10, further comprising:

12. An image decoding device is characterized by comprising an analysis module, a prediction module and a reconstruction module;

the analysis module is used for analyzing the code stream and acquiring a prediction mode of the current prediction unit and a residual error value between a true value and a predicted value of the current prediction unit; the prediction mode of the current prediction unit at least comprises a first prediction mode and a second prediction mode, wherein the first prediction mode is used for predicting a first part of the current prediction unit, the second prediction mode is used for predicting a second part of the current prediction unit, and the first prediction mode is different from the second prediction mode;

the prediction module is configured to predict a first portion of the current prediction unit by using the first prediction mode, and predict a second portion of the current prediction unit by using the second prediction mode, so as to obtain a predicted value of the current prediction unit;

the reconstruction module is used for determining a reconstruction value of the current prediction unit according to the prediction value of the current prediction unit and the residual error value of the current prediction unit.

13. An image decoding device is characterized by comprising an analysis module, a prediction module and a reconstruction module;

The analysis module is used for analyzing the code stream and acquiring a prediction mode of the current prediction unit and a residual error value between a predicted value and a true value of the current prediction unit;

the prediction module is used for predicting the current prediction unit by adopting a prediction mode of the current prediction unit to obtain a prediction value of the current prediction unit;

14. An image decoder for performing the image decoding method according to any one of claims 1 to 11.

15. An electronic device comprising an image decoder, a communication interface, and a memory; the image decoder is used for the image decoding method according to any one of claims 1 to 11.

16. A computer readable storage medium comprising computer instructions which, when run on an electronic device, cause the electronic device to perform the image decoding method of any one of claims 1 to 11.