CN117915082A

CN117915082A - Image encoding method, image decoding device, and computer storage medium

Info

Publication number: CN117915082A
Application number: CN202311746442.6A
Authority: CN
Inventors: 彭双; 江东; 林聚财; 方诚; 张雪; 殷俊
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2023-12-18
Filing date: 2023-12-18
Publication date: 2024-04-19

Abstract

The application discloses an image encoding method, an image decoding method, an image encoding device, an image decoding device and a computer storage medium, wherein the image encoding method comprises the following steps: acquiring an application scene of an image to be encoded; determining a layered coding mode according to the application scene; acquiring a reference relation type of an image to be coded according to the hierarchical coding mode; acquiring a reference image of the image to be coded based on the reference relation type; and coding the image to be coded by using the reference image to obtain a coded code stream of the image to be coded. The image coding method adopts a multi-mode layered coding method, can support flexible layered mode selection, and is compatible with different application scenes.

Description

Image encoding method, image decoding device, and computer storage medium

Technical Field

The present application relates to the field of image encoding technology, and more particularly, to an image encoding method, an image decoding method, an image encoding apparatus, an image decoding apparatus, and a computer storage medium.

Background

The video image data size is relatively large, and video pixel data (RGB, YUV, etc.) is usually required to be compressed, and the compressed data is called a video code stream, and the video code stream is transmitted to a user terminal through a wired or wireless network and then decoded and watched. The whole video coding flow comprises the processes of block division, prediction, transformation, quantization, coding and the like.

However, in the prior art, spatial hierarchy coding supports inter-layer prediction, and since the scene assumed by spatial hierarchy coding is a scene using multiple resolutions, only a certain resolution may be needed in practical application, and this scene will result in an increase in bandwidth and storage burden, for example, 720p video needs 1M bandwidth and 1080p video needs 2M bandwidth. If layered coding is used, the inter-layer prediction can use inter-layer relation to improve compression efficiency, and if 720p+1080 needs 2.6M bandwidth in total, but in practical application, 1080p video is needed only, 2.6M bandwidth will be needed by using layered coding, and instead of layered coding, only 2M bandwidth is needed. The existing spatial domain layered coding mode cannot select an optimal layered coding mode aiming at an application scene, so that the layered coding scheme is inflexible to use.

Disclosure of Invention

The application provides an image encoding method, an image decoding method, an image encoding device, an image decoding device and a computer storage medium.

The application adopts a technical scheme that an image coding method is provided, and the image coding method comprises the following steps:

acquiring an application scene of an image to be encoded;

determining a layered coding mode according to the application scene;

acquiring a reference relation type of an image to be coded according to the hierarchical coding mode;

Acquiring a reference image of the image to be coded based on the reference relation type;

And coding the image to be coded by using the reference image to obtain a coded code stream of the image to be coded.

The image to be coded comprises a first image to be coded and a second image to be coded, which are needed to be coded in sequence, wherein the first image to be coded and the second image to be coded are located in different image levels;

The obtaining the reference image of the image to be coded based on the reference relation type comprises the following steps:

Responding to the fact that the level of the first image to be coded is lower than the level of the second image to be coded, and judging whether an interlayer reference relation exists between the level of the first image to be coded and the level of the second image to be coded;

if not, setting a reference restriction relation of the first image to be coded according to the restriction that the first image to be coded cannot carry out intra-layer reference on the images in the image sequence of the second image to be coded and the images in the subsequent image sequence;

Acquiring a reference image of the first image to be coded based on the reference relation type of the first image to be coded and the reference restriction relation of the first image to be coded;

Responding to the level of the first image to be coded being higher than the level of the second image to be coded, and setting a reference limit relation of the first image to be coded according to the fact that the first image to be coded cannot carry out intra-layer reference limitation on images in the image sequence of the second image to be coded and the images in the subsequent image sequence;

wherein the image sequence includes an image display sequence and/or an image decoding sequence.

Wherein, in response to the first image to be encoded being lower in level than the second image to be encoded, the reference constraint relationship of the second image to be encoded is: intra-layer referencing cannot be performed on the image in the image sequence in which the first image to be encoded is located and the previous image sequence;

Responding that the layer level of the first image to be coded is higher than the layer level of the second image to be coded, and no interlayer reference relation exists between the layer level of the first image to be coded and the layer level of the second image to be coded, wherein the reference restriction relation of the second image to be coded is as follows: intra-layer reference cannot be made to the picture in which the first picture to be encoded is located and in which the previous picture is located.

The second image to be coded comprises a plurality of third images to be coded, wherein the third images to be coded are images to be coded of different image levels in the image sequence of the second image to be coded;

the determining whether the layer level of the first image to be encoded and the layer level of the second image to be encoded have an interlayer reference relationship includes:

Judging whether an interlayer reference relation exists between a level where the first image to be coded is located and a level where the plurality of third images to be coded are located;

if yes, an interlayer reference relation exists between the level of the first image to be coded and the level of the second image to be coded;

If not, the layer reference relation does not exist between the layer where the first image to be coded is located and the layer where the second image to be coded is located.

The first image to be coded comprises a plurality of fourth images to be coded, wherein the fourth images to be coded are images to be coded of image levels with different image sequences of the first image to be coded;

And in response to no interlayer reference relationship exists between the level of the second image to be coded and any one of the levels of the fourth images to be coded, the reference constraint relationship of the second image to be coded is: intra-layer reference cannot be made to the picture in which the first picture to be encoded is located and in which the previous picture is located.

The image to be encoded comprises a fifth image to be encoded, a sixth image to be encoded, a seventh image to be encoded and an eighth image to be encoded, which are required to be encoded in sequence, wherein the fifth image to be encoded and the sixth image to be encoded are located at different image levels, and the seventh image to be encoded and the eighth image to be encoded are located at different image levels;

the image encoding method further includes:

And setting a reference restriction relation of the fifth image to be coded according to the restriction that the fifth image to be coded cannot carry out intra-layer reference on the images in the image sequence of the sixth image to be coded and the images in the subsequent image sequence.

And setting a reference restriction relation of the sixth to-be-encoded image according to the restriction that the sixth to-be-encoded image cannot carry out intra-layer reference on the image sequence of the fifth to-be-encoded image and the image of the previous image sequence.

Wherein, the image coding method further comprises:

Setting a reference restriction relation of the seventh to-be-encoded image according to the fact that the seventh to-be-encoded image cannot carry out intra-layer reference restriction on the image in which the fifth to-be-encoded image is positioned and the previous image sequence, and the image in which the eighth to-be-encoded image is positioned and the subsequent image sequence;

And setting the reference restriction relation of the eighth to-be-encoded image and the image with the image sequence after the eighth to-be-encoded image according to restriction setting that the image with the image sequence before the eighth to-be-encoded image cannot be subjected to intra-layer reference.

Wherein, the image coding method further comprises:

Setting a switch syntax according to whether a multi-mode layered coding scheme and/or a layered switching scheme is enabled;

the scheme syntax is set according to the initiated multi-mode layered coding scheme and/or layered switching scheme.

Another technical solution adopted by the present application is to provide an image decoding method, the image decoding method comprising:

Acquiring an image code stream, and a layered coding mode and a reference relation type of the image code stream;

And decoding the image code stream according to the hierarchical coding mode and the reference relation type to obtain a reconstructed image of the image code stream.

Another technical scheme adopted by the application is to provide an image coding device, which comprises a memory and a processor coupled with the memory;

Wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the image encoding method as described above.

Another technical scheme adopted by the application is to provide an image decoding device, which comprises a memory and a processor coupled with the memory;

wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the image decoding method as described above.

Another aspect of the present application is to provide a computer storage medium for storing program data, which when executed by a computer, is used to implement the image encoding method and/or the image decoding method as described above.

The beneficial effects of the application are as follows: the image coding device acquires an application scene of an image to be coded; determining a layered coding mode according to the application scene; acquiring a reference relation type of an image to be coded according to the hierarchical coding mode; acquiring a reference image of the image to be coded based on the reference relation type; and coding the image to be coded by using the reference image to obtain a coded code stream of the image to be coded. The image coding method adopts a multi-mode layered coding method, can support flexible layered mode selection, and is compatible with different application scenes.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.

FIG. 1 is a schematic flow chart of a coding and decoding method provided by the application;

FIG. 2 is a schematic representation of spatial stratification provided by the present application;

FIG. 3 is a schematic overall flow diagram of a multi-mode layered coding scheme provided by the present application;

FIG. 4 is a flowchart of an embodiment of an image encoding method according to the present application;

FIG. 5 is a schematic diagram of a reference relationship of different hierarchical coding modes employed by different levels provided by the present application;

FIG. 6 is a flowchart of another embodiment of an image encoding method according to the present application;

FIG. 7 is a schematic diagram of an up-switch reference restriction without interlayer reference relationship between an initial layer and a target layer according to the present application;

FIG. 8 is a schematic diagram illustrating an up-switch reference restriction for an inter-layer reference relationship between an initial layer and a target layer provided by the present application;

FIG. 9 is a schematic diagram of an up-switch reference restriction provided by the present application when multiple target layers are present;

FIG. 10 is a schematic diagram illustrating a down-switch reference restriction for an initial layer and a target layer without interlayer reference relation;

FIG. 11 is a schematic diagram illustrating a down-switch reference restriction for an inter-layer reference relationship between an initial layer and a target layer provided by the present application;

FIG. 12 is a schematic diagram of a reference restriction for down-switch of multiple initial layers to the same target layer provided by the present application;

FIG. 13 is a schematic illustration of the simultaneous presence of an upward reference limit and a downward reference limit provided by the present application;

FIG. 14 is a flowchart of another embodiment of an image encoding method according to the present application;

FIG. 15 is a schematic view of a reference restriction provided by the present application when there are multiple switch points;

FIG. 16 is a schematic diagram of the reference relationship of frame level hierarchical switch + up and down hierarchical switch + arbitrary level switch provided by the present application;

FIG. 17 is a schematic diagram of a reference relationship of sequential level switching+up and down level switching+arbitrary level switching provided by the present application;

FIG. 18 is a flowchart of an embodiment of an image decoding method according to the present application;

FIG. 19 is a schematic diagram of an embodiment of an image encoding apparatus according to the present application;

Fig. 20 is a schematic diagram of an embodiment of an image decoding apparatus according to the present application;

Fig. 21 is a schematic structural diagram of an embodiment of a computer storage medium according to the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The overall process of encoding and decoding, the encoding process starts from the input video frame to the end of the code stream, and the decoding process starts from the code stream to the end of the reconstructed frame, as shown in fig. 1, fig. 1 is a schematic flow diagram of the encoding and decoding method provided by the application. Wherein, the dotted line is the coding and decoding common flow.

In video coding, the most commonly used color coding methods include YUV, RGB, etc., and the color coding method adopted in the application is YUV. Y represents brightness, that is, a gray value of an image; u and V (i.e., cb and Cr) represent chromaticity, which functions to describe image color and saturation. Each Y luminance block corresponds to one Cb and one Cr chrominance block, and each chrominance block corresponds to only one luminance block. Taking a 4:2:0 sampling format as an example, a block of n×m corresponds to a luminance block of size n×m, the corresponding two chrominance blocks are both of size (N/2) ×m/2, and the chrominance block is 1/4 of the luminance block. For a 4:4:4 sampling format, the luma block is the same size as the chroma block.

Block division: in video Coding, an image frame is input, but when a frame of image is coded, it is necessary to divide a frame into several LCUs (Largist Coding Unit, maximum Coding units), then divide each Coding Unit into recursive CUs (Coding units) with different sizes, and video Coding is performed by using a CU as a Unit. Wherein the smallest coding unit is called SCU (Smallest Coding Unit).

Intra/inter prediction: in general, luminance and chrominance signal values of adjacent pixels are relatively close, have strong correlation, and if luminance and chrominance information is directly represented by the number of samples, more spatial redundancy exists in the data. If the redundant data is removed before encoding, the average number of bits representing each pixel is reduced, i.e., the data is compressed with reduced spatial redundancy.

And (3) transformation: and subtracting the actual value from the predicted value of the current block after the current block is predicted, so as to obtain a residual block, wherein the residual block represents the difference between the actual image and the predicted image of the current block. The residual block is then transformed, e.g., using DCT, DST, etc. Since for most images there are many flat areas and areas with slow content transformation, and the correlation between adjacent pixels is strong, by transformation, these correlations can be reduced, and the scattered distribution of the image energy in the spatial domain can be converted into a relatively concentrated distribution in the transformation domain, so that the spatial redundancy can be removed.

Quantification: quantization is a process of mapping continuous values of a signal into a plurality of discrete amplitude values, and the mapping of the signal values into one-to-many values is realized. After the residual data is transformed, the transformation coefficient has a larger value range, and the quantization can effectively reduce the value range of the signal, so that a better compression effect is obtained. Quantization is the root cause of image distortion, since quantization is the dispersion of successive values into individual quantization intervals.

Spatial domain hierarchical coding: in spatial hierarchy coding, the content of each image sequence may be divided into multiple levels according to resolution, as shown in fig. 2, and fig. 2 is a schematic diagram of spatial hierarchy provided in the present application. The resolution of each level image may be the same or different, and in general, the resolution of a higher level image cannot be lower than the resolution of a lower level image, the lowest level (level 0) is generally referred to as a base layer, and the other levels are referred to as enhancement layers, N > =2.

The application provides a multi-mode hierarchical coding scheme, an overall flow chart of the method is shown in fig. 3, and fig. 3 is an overall flow chart of the multi-mode hierarchical coding scheme provided by the application, and the multi-mode hierarchical coding scheme comprises three parts, namely a multi-mode hierarchical coding scheme, a hierarchical switching scheme, scheme application and syntax expression.

As shown in fig. 3, the multi-mode layered coding scheme is: a hierarchical coding scheme is designed based on the reference relationship type. The layered switching scheme is as follows: in layered coding, in order to satisfy flexible layered switching, how the reference relationship of layered coding should be limited. The scheme application and the syntax expression are as follows: how to apply and how to express syntax for multi-mode layered coding and layered switching schemes.

The multi-mode hierarchical coding scheme provided by the application comprises the following steps:

1) Classification of reference relationships: the reference relationships of layered coding are classified into various types.

2) Multi-mode layering scheme design: different hierarchical coding modes are designed based on the reference relationship type.

In hierarchical coding, reference relationship types can be classified into a plurality of types, and can be classified into:

1) Intra-layer reference: when the reference relationship between the same hierarchy, such as the nth hierarchy image refers to the nth hierarchy image, as shown by the black solid arrow in fig. 3, the image sequence t is schematically represented by the intra-layer reference relationship of different hierarchy, and the reference relationship is intra-layer reference. FIG. 3 is a schematic diagram of a hierarchical reference relationship provided by the present application.

2) Interlayer reference: when the reference relationship between different levels, such as the ith level image refers to the jth level image, as indicated by the dashed arrow in fig. 3, the reference relationship between different levels is referred to as inter-layer reference, and in this case, the inter-layer reference only allows the high level to refer to the low level, i.e., N-1> =i > j > =0 is required.

Further, the inter-layer reference may be divided into pictures according to whether the same picture order is referred to:

synchronous interlayer reference: when inter-layer reference is made, the coded picture is in the same picture order as the reference picture, as indicated by the dashed arrow a in fig. 3.

Asynchronous interlayer reference: when inter-layer reference is made, the coded picture is in a different picture order than the reference picture, as indicated by the dashed arrow b in fig. 3.

In the multi-mode hierarchical coding scheme, at least one hierarchical coding mode is supported in the codec, and the hierarchical coding modes used by different levels can be the same or different.

For a particular layered coding mode, which uses at least one reference type, the layered coding mode scheme includes, but is not limited to, the following:

1) Mode 0: in layered coding, only intra-layer references are allowed; in the mode, each layer has no dependency relationship, independent decoding can be performed, and when only a single-layer code stream is needed, the single-layer code stream can be directly extracted from all the code streams, so that unnecessary code streams can be prevented from being transmitted.

2) Mode 1: in layered coding, only inter-layer references are allowed, including synchronous inter-layer references and asynchronous inter-layer references.

3) Mode 2: in layered coding, only synchronous inter-layer reference is allowed.

4) Mode 3: in layered coding, intra-layer referencing and synchronous inter-layer referencing are allowed.

Based on the above-mentioned multi-mode hierarchical coding scheme, the present application provides a specific image coding method, please continue to refer to fig. 4, fig. 4 is a flow chart of an embodiment of the image coding method provided by the present application.

As shown in fig. 4, the image encoding method according to the embodiment of the present application includes the steps of:

step S11: and acquiring an application scene of the image to be encoded.

In the embodiment of the present application, the application scenario related to the present application includes, but is not limited to: a single video content scene, multiple resolution scenes, and multiple video content scenes.

Step S12: and determining a layered coding mode according to the application scene.

In the embodiment of the application, the image coding device automatically selects the layered coding mode according to the application scene, and the corresponding relation between the application scene and the layered coding mode can be pre-configured manually.

In a specific first embodiment, all levels of hierarchical coding employ the same hierarchical coding mode, with coding modes supporting modes 0 and 3.

When the application scene only needs single video content, such as video storage, only needs to store the content of the high resolution level, and mode 0 is selected.

When multiple resolutions or multiple video contents are required in the application scene, mode 3 is selected.

For a scene of multiple resolutions: in the case of fluctuation of network bandwidth, if the current bandwidth is insufficient, only a low-quality or low-resolution level in the transmission levels is needed, if the current bandwidth is sufficient, a high-quality or high-resolution level needed for transmission is needed, and because of interlayer dependence, a high-level decoding needs to depend on the low level, and at the moment, the low level needs to be transmitted together.

Scenes for multiple video content: for example, when hierarchical coding is used for privacy protection, the base layer is used for coding non-private images, and the enhancement layer is used for coding private images, in which case the user may only need to decode the non-private images or may need to decode the private images.

In a specific second embodiment, the hierarchical structure includes both hierarchical coding and privacy protection, as shown in fig. 5, where level 0 is used to code low resolution non-private images, level 1 is used to code low resolution private images, level 2 is used to code high resolution non-private images, and level 3 is used to code high resolution private images.

In this embodiment, different hierarchical coding modes are employed by different levels, level 1 and level 3 employing mode 3, i.e. intra prediction+synchronous inter-layer prediction, and level 2 and level 0 employing mode 0, i.e. intra prediction only. Wherein, the synchronous inter-layer prediction of the level 1 only allows reference to the level 0, and the synchronous inter-layer prediction of the level 3 only allows reference to the level 2.

Step S13: and obtaining the reference relation type of the image to be coded according to the hierarchical coding mode.

In the embodiment of the present application, the image encoding device determines the hierarchical reference relationship type of the image to be encoded according to the hierarchical encoding modes determined in step S12, and the hierarchical reference relationship types allowed by each hierarchical encoding mode are described in detail in the above description, which is not repeated here.

Step S14: and acquiring a reference image of the image to be coded based on the reference relation type.

In the embodiment of the application, the image encoding device acquires the reference image of the image to be encoded according to the hierarchical reference relationship type determined in step S13. For example, if the reference relationship type is intra-layer reference, the reference picture of the picture to be encoded is a picture of other picture order of the same picture level. If the reference relation type is synchronous interlayer reference, the reference images of the images to be coded are images with the same image sequence of different image levels. If the reference relation type is asynchronous interlayer reference, the reference image of the image to be coded is an image of other image sequences of different image levels.

Step S15: and coding the image to be coded by using the reference image to obtain a coding code stream of the image to be coded.

In the embodiment of the application, an image coding device acquires an application scene of an image to be coded; determining a layered coding mode according to the application scene; acquiring a reference relation type of an image to be coded according to the hierarchical coding mode; acquiring a reference image of the image to be coded based on the reference relation type; and coding the image to be coded by using the reference image to obtain a coded code stream of the image to be coded. The image coding method adopts a multi-mode layered coding method, can support flexible layered mode selection, and is compatible with different application scenes.

Further, in the existing spatial hierarchy coding, flexible hierarchy switching cannot be supported, that is, it is difficult to switch to other hierarchy when video of a certain hierarchy is viewed in decoding.

In this regard, the present application solves the video real-time level switching problem through the layered switching scheme shown in fig. 3. Specifically, in hierarchical coding, in order to perform level switching in some image sequences, that is, the decoding level before level switching is m, and the decoding level after level switching is m ', m+.m', some reference relationships in hierarchical coding need to be restricted.

Generally, the level switching can be divided into two directions, the initial level is larger than the target level, the initial level is lower than the target level, the up-switching is performed, the initial level is the level before switching, and the target level is the level after switching.

1) Up-switch reference restriction: in the target layer and its inter-layer dependent layers, which do not include the start layer, the switching point and subsequent pictures cannot be intra-layer referenced across the switching point, i.e. forward reference restriction. If the target layer and the initial layer have direct or indirect interlayer references, the internal references of the initial layer and the interlayer dependent layers thereof are not limited; otherwise, in the initial layer and its inter-layer dependent layers, the picture before the switching point cannot be intra-layer referenced across the switching point, i.e. backward referencing restriction.

Wherein, the inter-layer dependency layer refers to a layer level directly and indirectly performing inter-layer reference, and if any image sequence in the sequence has direct or indirect inter-layer reference, the inter-layer dependency is considered to exist; if there are multiple possible target layers, the starting layer needs to meet the restrictions on the starting layer among all the possible target layers.

In particular, when there are a plurality of switching points, the switching point in the forward reference restriction is the switching point image closest to and located in front in the image display order; the switching point in the backward reference restriction is the switching point image closest in image display order and located later.

2) Down-switch reference limit: in the starting layer and its inter-layer dependent layers, which do not include the target layer, the switching point and the previous picture cannot be intra-layer referenced across the switching point, i.e. backward referencing restriction. If the direct or indirect interlayer reference exists between the initial layer and the target layer, the intra-layer reference of the target layer and the interlayer dependent layer is not limited; otherwise, in the target layer and the inter-layer dependency layer thereof, the switching point and the following images cannot perform intra-layer reference across the switching point, namely, forward reference restriction. If there are multiple starting layers that can be switched to the same target layer, the target layer needs to meet the limitation of all the starting layers to the target layer.

With continued reference to fig. 6, fig. 6 is a flowchart illustrating an image encoding method according to another embodiment of the present application.

As shown in fig. 6, the image encoding method of the embodiment of the present application includes the steps of:

Step S21: and responding to the level of the first image to be coded being lower than the level of the second image to be coded, and judging whether an interlayer reference relation exists between the level of the first image to be coded and the level of the second image to be coded.

In the embodiment of the application, the first image to be encoded and the second image to be encoded are images to be encoded in adjacent image sequence, wherein the first image to be encoded is in the preceding image sequence, and the second image to be encoded is in the following image sequence. Therefore, the level of the first image to be encoded is the initial layer, and the level of the second image to be encoded is the target layer.

The image coding device judges whether a direct or indirect interlayer reference relation exists between the level of the first image to be coded and the level of the second image to be coded. If the inter-layer reference relationship does not exist, the process proceeds to step S22.

If there is an inter-layer reference relationship, and the layer level of the first image to be encoded is higher than the layer level of the second image to be encoded, the intra-layer reference of the layer level of the second image to be encoded and the inter-layer dependent layer thereof is not limited.

If the level of the first image to be coded is lower than the level of the second image to be coded, the inter-layer reference relation is not required to be considered, and the reference restriction relation of the second image to be coded is as follows: intra-layer reference cannot be made to the picture in which the first picture to be encoded is in and in which the previous picture is in.

Step S22: and setting a reference restriction relation of the first image to be coded according to the restriction that the first image to be coded cannot carry out intra-layer reference on the images in the image sequence of the second image to be coded and the images in the subsequent image sequence.

In the embodiment of the present application, when the level of the first image to be encoded is higher than the level of the second image to be encoded, the reference constraint relationship of the first image to be encoded is: the intra-layer reference cannot be performed on the image in the image sequence in which the second image to be encoded is located and the subsequent image sequence.

When the level of the first image to be coded is lower than the level of the second image to be coded and an interlayer reference relation exists between the level of the first image to be coded and the level of the second image to be coded, the reference restriction relation of the first image to be coded is as follows: the intra-layer reference cannot be performed on the image in the image sequence in which the second image to be encoded is located and the subsequent image sequence.

In a specific third embodiment, assuming that there is a four-layer spatial hierarchy, at the switch point, a switch is made from level 0 to level 2, the initial layer is level 0, and the target layer is level 2. Referring to fig. 7 and 8, fig. 7 is a schematic view illustrating an up-switch reference restriction where the starting layer and the target layer have no interlayer reference relationship, and fig. 8 is a schematic view illustrating an up-switch reference restriction where the starting layer and the target layer have an interlayer reference relationship.

In the reference restriction diagram of the present application, a broken line arrow 1 indicates a hierarchical switch, broken lines and an x arrow indicate reference restrictions, a broken line arrow 2 indicates interlayer references, a solid line arrow indicates a sequential image display flow of each image sequence, and a filled image frame indicates a decoded image.

As shown in fig. 7, there is an inter-layer reference between level 2 and level 1, and there is no inter-layer reference relationship between level 2 and level 0. As shown in fig. 8, there is an inter-layer reference between level 2 and level 1, an inter-layer reference between level 1 and level 0, and an indirect inter-layer reference relationship between level 2 and level 0.

As shown in fig. 7, when the target layer and the start layer have no inter-layer reference relationship, the target layer and the inter-layer dependent layer, i.e., the intra-layer references of level 2 and level 1, cannot refer to the image before the switching point. Meanwhile, in the intra-layer references of the start layer and its dependent layers, the image before the switching point cannot refer to the image after the switching point.

As shown in fig. 8, when the target layer and the start layer have an inter-layer reference relationship, the target layer and the inter-layer dependent layer, i.e., the intra-layer references of level 2 and level 1, cannot refer to the image before the switching point. Wherein the inter-layer dependency layer of the target layer includes level 0 but level 0 is excluded as the starting layer. At this time, there is no reference limitation in the intra-layer references of the starting layer and its dependent layers.

Further, when there are multiple possible target layers, as shown in fig. 9, fig. 9 is a schematic view of the reference restriction of up-switch when there are multiple target layers provided in the present application. The left diagram of fig. 9 switches from level 0 to level 1, where the intra-layer references for level 0 and its dependent layers are not limited. In fig. 9, the layer frame 0 is switched to the layer 2, and at this time, for intra-layer references of the layer frame 0 and its dependent layers, the image before the switching point cannot refer to the image after the switching point.

Therefore, in order to satisfy the level switching of the left and right graphs at the same time, for intra-layer references of level 0 and its dependent layers, images before the switching point cannot refer to the switching point and images after it.

In a specific fourth embodiment, assuming that there is a spatial hierarchy of four layers, at the switching point, switching from level 2 to level 0, the initial layer is level 2, and the target layer is level 0, as shown in fig. 10 and fig. 11, where fig. 10 is a schematic view of a down-switch reference restriction provided by the present application, in which the initial layer and the target layer have no inter-layer reference relationship, and fig. 11 is a schematic view of a down-switch reference restriction provided by the present application, in which the initial layer and the target layer have an inter-layer reference relationship.

In fig. 10, there is an inter-layer reference between level 2 and level 1, and there is no inter-layer reference relationship between level 2 and level 0. In fig. 11, there is an inter-layer reference between level 2 and level 1, an inter-layer reference between level 1 and level 0, and an indirect inter-layer reference relationship between level 2 and level 0.

As shown in fig. 10, when the starting layer and the target layer have no inter-layer reference relationship, the image before the switching point cannot refer to the switching point and the image after the switching point in the intra-layer references of the starting layer and the inter-layer dependent layers, i.e., the layer 2 and the layer 1. Meanwhile, in the intra-layer references of the target layer and its dependent layers, the switching point and the subsequent image cannot refer to the image before the switching point.

As shown in fig. 11, when the inter-layer reference relationship exists between the start layer and the target layer, the pictures before the switch point cannot refer to the switch point and the pictures after the switch point in the inter-layer references of the start layer and the inter-layer dependent layers, i.e., the layer references of the layer 2 and the layer 1. Wherein the inter-layer dependency layer of the starting layer includes level 0, but level 0 is excluded as the target layer. At this time, there is no reference restriction in intra-layer references of the target layer and its dependent layers.

Further, when there are multiple starting layers that can be switched to the same target layer, as shown in fig. 12, fig. 12 is a schematic view of a reference restriction of down-switch provided by the present application. The left diagram of fig. 12 switches from level 1 to level 0, where the intra-layer references for level 0 and its dependent layers are not limited. In the right diagram of fig. 12, the layer frame 2 is switched to the layer frame 0, and at this time, the image at the switching point and after cannot refer to the image before the switching point in the intra-layer references of the layer frame 0 and the dependent layers thereof. Therefore, in order to satisfy the level switching of the left and right graphs at the same time, for the intra-layer references of the level 0 and its dependent layers, the image before the switching point cannot be referenced for the image after the switching point.

In a specific fifth embodiment, it is assumed that there is a spatial hierarchy of two layers and for privacy protection, where level 0 is used to encode non-private images, level 1 is used to encode private images, and a synchronization of inter-layer references is allowed between level 1 and level 0. As shown in fig. 13, fig. 13 is a schematic diagram of the simultaneous presence of upward and downward reference limits provided by the present application.

When there is both an up and down level handoff at the handoff point. In the down-hierarchy switching, i.e., the hierarchy 1 is switched to the hierarchy 0, as shown in the left diagram of fig. 13, the intra-layer reference of the hierarchy 0 is not limited, and the intra-layer reference of the hierarchy 1 is limited to the image before the switching point cannot refer to the switching point and the image after the switching point.

In the up-hierarchy switching, i.e., the hierarchy 0 is switched to the hierarchy 1, as shown in the right diagram of fig. 13, the intra-layer reference restriction of the hierarchy 1 is that the switching point and the subsequent image cannot refer to the image before the switching point, and the hierarchy 0 has no reference restriction. Therefore, at this time, there is no restriction on the intra-layer reference that level 0 does not exist, and the intra-layer reference of level 1 is restricted in that the image before the switching point cannot refer to the switching point and the image after the switching point, while the switching point and the image after the switching point cannot refer to the image before the switching point.

Step S23: a reference picture of the first picture to be encoded is obtained based on the reference relation type of the first picture to be encoded and the reference restriction relation of the first picture to be encoded.

In the embodiment of the present application, when the image encoding device acquires the reference image of the first image to be encoded according to the reference relationship type of the first image to be encoded, the image encoding device needs to conform to the reference restriction relationship of the first image to be encoded. Similarly, when the image encoding device acquires the reference image of the second image to be encoded according to the reference relationship type of the second image to be encoded, the image encoding device also needs to conform to the reference restriction relationship of the second image to be encoded.

In the above layered switching scheme, when there are two or more switching points, the present application further proposes another image encoding method, please continue to refer to fig. 14, fig. 14 is a flowchart of another embodiment of the image encoding method provided by the present application.

Step S31: and setting a reference restriction relation of the fifth image to be encoded according to the restriction that the fifth image to be encoded cannot carry out intra-layer reference on the images in the image sequence of the sixth image to be encoded and the images in the subsequent image sequence.

In the embodiment of the present application, there are a plurality of switching points, for example, when the sixth image to be encoded and the eighth image to be encoded are both switching points, the reference constraint relationship of the fifth image to be encoded is as follows: the intra-layer reference cannot be performed on the image in the image sequence in which the sixth image to be encoded is located and the subsequent image sequence.

Step S32: and setting a reference restriction relation of the sixth to-be-encoded image according to the restriction that the sixth to-be-encoded image cannot carry out intra-layer reference on the image in the image sequence of the fifth to-be-encoded image and the image in the previous image sequence.

The reference constraint relationship of the sixth image to be encoded is as follows: the intra-layer reference cannot be performed on the picture in which the fifth picture to be encoded is located and in which the previous picture is located.

Step S33: and setting a reference restriction relation of the seventh to-be-encoded image according to the fact that the seventh to-be-encoded image cannot carry out intra-layer reference restriction on the image in which the fifth to-be-encoded image is positioned and the image in which the eighth to-be-encoded image is positioned.

The reference constraint relationship of the seventh image to be encoded is as follows: the intra-layer reference cannot be performed on the image in which the fifth image to be encoded is located and the image in which the previous image is located, and the image in which the eighth image to be encoded is located and the image in which the subsequent image is located.

Step S34: and setting the reference restriction relation of the eighth to-be-encoded image and the image in the image sequence after the eighth to-be-encoded image according to the restriction setting that the image in the image sequence before the eighth to-be-encoded image cannot be subjected to intra-layer reference.

The reference restriction relation of the eighth image to be encoded and the images of the image sequence thereafter is as follows: intra-layer referencing of the picture of picture order preceding said eighth picture to be encoded is not possible.

In a specific sixth embodiment, when there are two switching points, as shown in fig. 15, fig. 15 is a reference restriction diagram provided in the present application when there are a plurality of switching points. For the image before the switching point 1, when the level 1 performs backward reference, the switching point closest to and located at the rear is displayed as the switching point 1, and cannot cross the switching point 1, as shown in the left diagram of fig. 15 below. For the images between the switching point 1 and the switching point 2, including the switching point 1 and excluding the switching point 2, when the hierarchy 1 refers to the front, the switching point closest to and located in front is displayed as the switching point 1, and cannot cross the switching point 1, as shown in the right graph of fig. 15 below. When the level 1 is referred to in the backward direction, the switching point closest to and located at the rear is shown as the switching point 2, and cannot cross the switching point 2, as shown in the left diagram of fig. 15 below, and when the level 1 is referred to in the forward direction, the switching point closest to and located at the rear is shown as the switching point 2, and cannot cross the switching point 2, as shown in the right diagram of fig. 15 below.

It should be noted that, in the above embodiment, the image before the switching point 1 is the fifth image to be encoded, and the switching point 1 is the sixth image to be encoded; the image between the switching point 1 and the switching point 2 is the seventh image to be encoded, and the image of the switching point 1 is the eighth image to be encoded.

Further, continuing with the description of the schema application and syntax representation in FIG. 3:

The scheme application of the application specifically comprises multi-mode hierarchical coding application and hierarchical switching scheme application.

In particular, multi-mode layered coding applications include, but are not limited to, two multi-mode schemes, i.e., different levels using the same layered coding mode, different levels using different layered coding modes. The first embodiment described above describes using the same layered coding mode for different levels; the second embodiment described above describes the use of different hierarchical coding modes for different levels.

The layered switching scheme application proposed by the present application is based on switching granularity, including but not limited to:

1) Frame-level hierarchical switching: in a video sequence, some pictures may be hierarchically switched.

2) Successive image layering switching: in a video sequence, some successive pictures may be hierarchically switched.

3) Sequence level hierarchical switching scheme: in a video sequence, all images may be hierarchically switched. The layered switching scheme application proposed by the present application is based on the switching direction, including but not limited to:

1) Only down hierarchical handover: in a hierarchical switching scheme, the hierarchy of the switching point images allows only the hierarchy to switch down.

2) Only up-hierarchical handover: in a hierarchical switching scheme, the hierarchy of the switch point images allows only the hierarchy to allow only up-switches.

3) Up and down hierarchical switching: in a hierarchical switching scheme, the hierarchy of switching point images may be switched up or down.

The hierarchical switching scheme application proposed by the present application is based on the span of the switching hierarchy, including but not limited to:

1) Arbitrary level switching: in a hierarchical switching scheme, the switch point image may be switched to any hierarchy.

2) Hierarchical switching: in a hierarchical switching scheme, a switching point image may be switched to an adjacent hierarchy.

The layered switching scheme provided by the application can freely combine the switching granularity, the switching direction and the switching level span to further form more layered switching schemes.

In particular: for the frame-level switching scheme, in order to meet the real-time requirement, the reference relationship needs to be limited: for any switched picture, the decoding order must precede the display order by pictures following the switched picture, such as a low-latency reference structure that allows only forward reference.

In a specific seventh embodiment, frame-level hierarchical switching+up and down hierarchical switching+arbitrary level switching: only pictures of the picture display order (POC) in the video sequence can be hierarchically switched, and each hierarchy can be hierarchically switched up and down, and from each hierarchy can be switched to any hierarchy according to the switching direction.

Referring specifically to fig. 16, fig. 16 is a schematic diagram of a reference relationship between frame-level hierarchical switching, up-and down-level hierarchical switching, and arbitrary-level switching according to the present application. As shown in fig. 16, there is a 2-level hierarchical coding, there is an inter-layer reference relationship between level 1 and level 0, and the image display sequence switch_poc at the switching point is 1, at which the level can be switched from level 1 to level 0, and from level 0 to level 1.

At this point, level 0 has no reference limit, and level 1's reference limit is: the image before the switching point cannot refer to the switching point and the image after the switching point, and the constraint 1 can be written as: cur_poc < switch_poc & & ref_poc > = switch_poc; the images at and after the switch point cannot refer to the image before the switch point, and constraint 2 can be written as: cur_poc > = switch_poc & & ref_poc < switch_poc; if the current image satisfies any constraint, no reference can be made. Wherein cur_poc, switch_poc, ref_poc represent the display order of the current picture, the display order of the switching picture, the picture display order of the reference picture of the current picture, respectively.

In particular: the reference relationship is assumed to be a low-delay reference structure that allows only forward reference, where constraint 1 is constantly not satisfied.

In a specific eighth embodiment, sequence level hierarchical switching + up and down hierarchical switching + arbitrary level switching: all pictures in the video sequence can be hierarchically switched and each level can be hierarchically switched up and down and from each level can be switched to any level depending on the switching direction.

Referring specifically to fig. 17, fig. 17 is a schematic diagram of a reference relationship between sequential hierarchical switching+upward and downward hierarchical switching+arbitrary hierarchical switching. As shown in fig. 17, there is a 2-level hierarchical coding, and there is an inter-layer reference relationship between level 1 and level 0, since each image point can perform level switching, there is no reference restriction on level 0 at this time, and there is no reference relationship between any two images of level 1.

The syntax representation applied to the privacy permission switching scheme of the present application specifically includes, but is not limited to:

1) Switching syntax: for expressing the use of a multi-mode-enabled layered coding scheme and/or a layered switching scheme, the switching syntax may be as follows: including but not limited to Video Parameter Sets (VPS), sequence Parameter Sets (SPS), picture Parameter Sets (PPS), picture Header (PH), supplemental Enhancement Information (SEI), extension information.

2) Scheme syntax: when the multimode layered coding scheme and/or the layered switching scheme is enabled by default or the switch syntax enables the multimode layered coding scheme and/or the layered switching scheme, syntax elements necessary for implementing the multimode layered coding scheme and/or the layered switching scheme are expressed, and these elements may be as follows: including but not limited to Video Parameter Sets (VPS), sequence Parameter Sets (SPS), picture Parameter Sets (PPS), picture Header (PH), supplemental Enhancement Information (SEI), extension information.

3) Syntax coding: the syntax coding mode includes, but is not limited to, advanced entropy coding, signed fixed length coding, unsigned fixed length coding, exponential golomb coding, and the like.

In a specific ninth embodiment, based on the first embodiment, all levels of hierarchical coding adopt the same hierarchical coding mode, whether multi-mode hierarchical coding is enabled or not can be expressed through a switch syntax, the coding modes support mode 0 and mode 3, and for a specific used mode, expression needs to be performed through a scheme syntax.

As shown in the following table, the multi-mode hierarchical coding switch syntax multi_mode_svc_flag, and the scheme syntax multi_mode_svc_idx for expressing the coding mode are transmitted in the sequence header.

Multi_mode_svc_flag=0 indicates that multi-mode layered coding is not enabled, and multi_mode_svc_flag=1 indicates that multi-mode layered coding is enabled; when multi_mode_svc_flag=1, multi_mode_svc_idx=0 represents usage mode 0, and multi_mode_svc_idx=1 represents usage mode 3.

In a specific tenth embodiment, on the basis of the first embodiment, both mode 0 and mode 3 may satisfy the reference restriction of the hierarchy switching by restricting the reference relation, for example, the hierarchy switching point information may be transmitted through the SEI under a hierarchy switching scheme of frame level hierarchy switching+up and down hierarchy switching+any hierarchy switching, as shown in the following table, where ssvc _frame_nesting_poc represents an image sequence in which hierarchy switching may be performed, and the image sequence includes, but is not limited to, an image display sequence, an image decoding sequence, and the like.

Assume that the load type index PayloadType of the SEI message is 19, as shown in the following table.

In a specific eleventh embodiment, the reference limitation of the level switching may be satisfied by limiting the reference relation on the basis of the above second embodiment, the level switching including frame level switching+up and down level switching+arbitrary level switching, sequence level switching+up and down level switching+arbitrary level switching, both level switching may transmit level switching point information through the SEI, as shown in the following table, a level switching scheme using sequence level switching+up and down level switching+arbitrary level switching when the sequence level switching flag ssvc _seq_nesting_flag=1, and an image sequence ssvc _frame_nesting_poc of a switching point of the frame level switching scheme is further transmitted when the sequence level switching flag ssvc _seq_nesting_flag=0.

In a twelfth embodiment, the above-described seventh embodiment is followed by layer coding using layer coding mode 3, and layer switching using a layer switching scheme of frame-level switching, up-and down-layer switching, and arbitrary layer switching.

At this time, the number of switch points of frame-level switching syntax frame_ ssvc _nesting_num, and the picture order value ssvc _nesting_poc [ i ] of each switch point are transmitted in the sequence header, and the syntax expression is as shown in the following table.

In a specific thirteenth embodiment, the layered coding adopts the layered coding mode 3 on the basis of the seventh embodiment described above, the layered switching adopts a layered switching scheme of frame-level switching+up-and-down-layered switching+arbitrary-layered switching, and the reference structure is limited to a low-delay reference structure allowing only forward reference.

At this time, whether the image is a switching image is transmitted in the image header as shown in the following table. Wherein, when frame_ ssvc _nesting_flag=1, the picture is a switching picture; when FRAME SSVC NESTING FLAG =0, the image is a non-switching image.

In a specific fourteenth embodiment, the hierarchical coding adopts the hierarchical coding mode 3 based on the eighth embodiment, and the hierarchical switching adopts a hierarchical switching scheme of sequential hierarchical switching+upward and downward hierarchical switching+arbitrary hierarchical switching.

At this point, whether the sequence level hierarchy switching scheme is enabled is transmitted in the sequence header as shown in the following table. When seq_ ssvc _nesting_flag=1, the sequence-level hierarchical switching scheme is started, and all images are switched images at the moment; when seq_ ssvc _nesting_flag=0, it means that the sequence-level hierarchical switching scheme is not enabled.

In a specific fourteenth embodiment, it is assumed that the three level switching schemes in the twelfth embodiment, the thirteenth embodiment, and the fourteenth embodiment are adopted at the same time.

At this time, the hierarchy switch syntax is used to express whether hierarchy switching is supported, and if hierarchy switching is supported, the hierarchy switching scheme mode is further transmitted, and then the related syntax of each mode is specifically transmitted, as shown in the following table. Wherein ssvc _nesting_enable_flag: the hierarchy-switching-enabling flag, ssvc _nesting_enable_flag=1, represents the switching-on hierarchy, and ssvc _nesting_enable_flag=0 represents the switching-off hierarchy;

ssvc _nesting_mode: the hierarchical switching mode ssvc _nesting_mode=2 indicates that the hierarchical switching scheme of the twelfth embodiment is used; ssvc _nesting_mode=1 represents the hierarchical switching scheme of the thirteenth embodiment; ssvc _nesting_mode=0 represents the hierarchical switching scheme of the fourteenth embodiment.

When ssvc _nesting_mode=1, a switching picture flag is further transmitted in the picture header PH as shown in the following table.

The application provides a multi-mode layered coding and decoding method, which comprises a multi-mode layered coding scheme, a layered switching scheme, scheme application and syntax expression, and can support flexible layered mode selection. The multi-mode hierarchical coding scheme comprises reference relation classification and multi-mode hierarchical coding scheme design. In the layered switching scheme, the layered switching scheme mainly describes the reference relation limitation during layered switching, and flexible layered switching can be provided. The scheme application and the syntax expression comprise a multi-mode layered coding scheme application, a layered switching scheme application and corresponding syntax design.

With continued reference to fig. 18, fig. 18 is a flowchart illustrating an embodiment of an image decoding method according to the present application.

As shown in fig. 18, the image decoding method of the embodiment of the present application includes the steps of:

step S41: and acquiring the image code stream, and the hierarchical coding mode and the reference relation type of the image code stream.

Step S42: and decoding the image code stream according to the hierarchical coding mode and the reference relation type to obtain a reconstructed image of the image code stream.

The above embodiments are only one common case of the present application, and do not limit the technical scope of the present application, so any minor modifications, equivalent changes or modifications made to the above matters according to the scheme of the present application still fall within the scope of the technical scheme of the present application.

With continued reference to fig. 19, fig. 19 is a schematic structural diagram of an image encoding device according to an embodiment of the application. The image encoding apparatus 500 of the embodiment of the present application includes a processor 51, a memory 52, an input-output device 53, and a bus 54.

The processor 51, the memory 52, and the input/output device 53 are respectively connected to the bus 54, and the memory 52 stores program data, and the processor 51 is configured to execute the program data to implement the image encoding method described in the above embodiment.

In an embodiment of the present application, the processor 51 may also be referred to as a CPU (Central ProcessingUnit ). The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general purpose processor, a digital signal processor (DSP, digital Signal processes), an application specific integrated circuit (ASIC, application SpecificIntegrated Circuit), a field programmable gate array (FPGA, field Programmable GateArray) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The general purpose processor may be a microprocessor or the processor 51 may be any conventional processor or the like.

With continued reference to fig. 20, fig. 20 is a schematic diagram of an image decoding apparatus according to an embodiment of the application. The image decoding apparatus 600 of the embodiment of the present application includes a processor 61, a memory 62, an input-output device 63, and a bus 64.

The processor 61, the memory 62, and the input/output device 63 are respectively connected to the bus 64, and the memory 62 stores program data, and the processor 61 is configured to execute the program data to implement the image decoding method described in the above embodiment.

The present application further provides a computer storage medium, and referring to fig. 21, fig. 21 is a schematic structural diagram of an embodiment of the computer storage medium provided by the present application, in which program data 71 is stored in the computer storage medium 700, and the program data 71, when executed by a processor, is used to implement the image encoding method and/or the image decoding method of the above embodiments.

Embodiments of the present application may be stored in a computer readable storage medium when implemented in the form of software functional units and sold or used as a stand alone product. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a read-only memory (ROM), a random access memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing description is only of embodiments of the present application, and is not intended to limit the scope of the application, and the equivalent structures or equivalent processes disclosed in the specification and the drawings are used in the same way or directly or indirectly in other related technical fields, which are also included in the scope of the application.

Claims

1. An image encoding method, characterized in that the image encoding method comprises:

acquiring an application scene of an image to be encoded;

determining a layered coding mode according to the application scene;

2. The image encoding method according to claim 1, wherein,

The image to be encoded comprises a first image to be encoded and a second image to be encoded which are required to be encoded in sequence, wherein the first image to be encoded and the second image to be encoded are positioned at different image levels;

3. The image encoding method according to claim 2, wherein,

In response to the first to-be-encoded image being lower in level than the second to-be-encoded image, the reference constraint relationship of the second to-be-encoded image is: intra-layer referencing cannot be performed on the image in the image sequence in which the first image to be encoded is located and the previous image sequence;

4. The image encoding method according to claim 2, wherein,

5. The image encoding method according to claim 2, wherein,

6. The image encoding method according to claim 1, wherein,

The image to be encoded comprises a fifth image to be encoded, a sixth image to be encoded, a seventh image to be encoded and an eighth image to be encoded, which are required to be encoded in sequence, wherein the fifth image to be encoded and the sixth image to be encoded are positioned at different image levels, and the seventh image to be encoded and the eighth image to be encoded are positioned at different image levels;

the image encoding method further includes:

Setting a reference restriction relation of the fifth image to be coded according to the restriction that the fifth image to be coded cannot carry out intra-layer reference on the images in the image sequence of the sixth image to be coded and the images in the subsequent image sequence;

7. The image encoding method according to claim 6, wherein,

The image encoding method further includes:

8. The image encoding method according to claim 1, wherein,

The image encoding method further includes:

9. An image decoding method, characterized in that the image decoding method comprises:

10. An image encoding device, comprising a memory and a processor coupled to the memory;

Wherein the memory is for storing program data and the processor is for executing the program data to implement the image encoding method as claimed in any one of claims 1 to 8.

11. An image decoding device, comprising a memory and a processor coupled to the memory;

wherein the memory is for storing program data and the processor is for executing the program data to implement the image decoding method of claim 9.

12. A computer storage medium for storing program data which, when executed by a computer, is adapted to carry out the image encoding method of any one of claims 1 to 8 and/or the image decoding method of claim 9.