CN114630113A

CN114630113A - Decoding and encoding method based on adaptive intra-frame refreshing mechanism and related equipment

Info

Publication number: CN114630113A
Application number: CN202110204286.5A
Authority: CN
Inventors: 魏亮; 陈方栋; 王莉
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2021-02-23
Filing date: 2021-02-23
Publication date: 2022-06-14
Anticipated expiration: 2041-02-23
Also published as: CN113873236B; CN113873236A; CN114630113B

Abstract

The embodiment of the application discloses a decoding and encoding method based on an adaptive intra-frame refreshing mechanism and related equipment, belonging to the field of video coding and decoding. In the embodiment of the present application, virtual boundary position mark information is newly added in a code stream of a current frame, and the virtual boundary position mark information can indicate a position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed region from an un-refreshed region in the current frame, so that when a coding end divides a forced intra-frame region of the current frame, the virtual boundary can be arbitrarily set based on actual requirements, and then the position of the virtual boundary is indicated by the virtual boundary position mark information, so that the flexibility of dividing the forced intra-frame region is improved, that is, the embodiment of the present application provides an adaptive intra-frame refresh mechanism for adaptively dividing the forced intra-frame region. Furthermore, various ways of loop filtering across the virtual boundary may be allowed, thereby improving the subjective quality of the image on the virtual boundary.

Description

Decoding and encoding method based on adaptive intra-frame refreshing mechanism and related equipment

Technical Field

The embodiment of the application relates to the field of video coding and decoding, in particular to a decoding and coding method based on an adaptive intra-frame refreshing mechanism and related equipment.

Background

The intra-frame refreshing mechanism is to disperse the code rate of the I frame to a plurality of P frames, thereby solving the problem of longer time for decoding the I frame due to overlarge code rate of the I frame. Specifically, a complete image frame is divided into N forced intra-frame regions based on a refresh period, and during encoding, N intra-frame refresh frames for the image frame are sequentially encoded, each intra-frame refresh frame includes one forced intra-frame region of the N forced intra-frame regions, the encoding mode of the N forced intra-frame regions is a forced intra-frame (intra) mode, and the encoding mode of other regions in each intra-frame refresh frame allows an inter-frame (inter) mode to be adopted, so that the code rate of each intra-frame refresh frame is not only reduced relative to the code rate of an I-frame, but also is relatively stable. The forced intra-frame area in each intra-frame refresh frame and the forced intra-frame area in the intra-frame refresh frame positioned in front of the intra-frame refresh frame in the refresh period are merged and then are called as the refreshed area of the intra-frame refresh frame, and the other areas except the refreshed area in the intra-frame refresh frame are called as the non-refreshed area. In order to facilitate a decoding end to determine which decoding mode is used for decoding, for any intra-frame refresh frame, the code stream needs to carry the mark information, so that when decoding, the refreshed area and the non-refreshed area of the intra-frame refresh frame are determined based on the mark information, and then which decoding mode is used for decoding is determined.

Disclosure of Invention

The embodiment of the application provides a decoding and encoding method and related equipment based on an adaptive intra-frame refreshing mechanism, which can improve the flexibility of dividing into forced intra-frame areas. The technical scheme is as follows:

in one aspect, a decoding method based on an adaptive intra refresh mechanism is provided, the method including:

receiving a code stream of a current frame, wherein the code stream of the current frame carries frame type mark information, and the frame type mark information indicates whether the current frame is an intra-frame refresh frame;

acquiring virtual boundary position mark information carried in the code stream under the condition that the frame type mark information indicates that the current frame is an intra-frame refreshing frame, wherein the virtual boundary position mark information indicates the position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed area from an un-refreshed area in the current frame;

determining an area position of the refreshed area and an area position of the non-refreshed area based on the virtual boundary position mark information;

and decoding the code stream of the current frame based on the area position of the refreshed area and the area position of the non-refreshed area.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, the virtual boundary position mark information includes first virtual boundary position mark information and second virtual boundary position mark information, where the first virtual boundary position mark information indicates a position of the virtual boundary in a horizontal direction of the current frame, and the second virtual boundary position mark information indicates a position of the virtual boundary in a vertical direction of the current frame;

wherein a position of the virtual boundary in the current frame in the vertical direction of the current frame is different from a position of a virtual boundary between a refreshed region and a non-refreshed region in an adjacent frame in the vertical direction of the adjacent frame, and the adjacent frame is a previous frame or a next frame of the current frame in a decoding order.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, the first virtual boundary position mark information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of the virtual boundary in the vertical direction of the current frame.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, the first virtual boundary position mark information is an image abscissa of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is an image ordinate of the virtual boundary in the vertical direction of the current frame;

the refreshed area is a rectangular area with coordinates (0, 0) at the upper left corner of the image as an origin, the width as the abscissa of the image and the height as the ordinate of the image, and the non-refreshed area is an area other than the refreshed area in the current frame.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in the horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in the vertical direction of the current frame and a height of the current frame image.

Based on the decoding method based on the adaptive intra refresh mechanism provided above, in a possible implementation manner, the width precision of the refreshed region is the width of one coding unit CU, and the height precision of the refreshed region is the height of the CU; alternatively, the first and second electrodes may be,

the width precision of the refreshed area is the width of a Largest Coding Unit (LCU), and the height precision of the refreshed area is the height of the LCU; alternatively, the first and second electrodes may be,

the width precision of the refreshed area is the width of an image slice, the height precision of the refreshed area is the height of the image slice, and the image slice comprises one or more LCUs.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, the refreshed region includes a forced intra-frame region and an inter-frame region;

decoding the code stream of the current frame based on the area position of the refreshed area and the area position of the non-refreshed area, including:

determining a region position of the forced intra region and a region position of the inter region based on a region position of the refreshed region and a refreshed region of a previous frame of the current frame in decoding order;

and decoding the code stream of the current frame based on the area position of the forced intra-frame area and the area position of the inter-frame area.

Based on the decoding method based on the adaptive intra refresh mechanism provided above, in a possible implementation manner, the inter region is a refreshed region of the previous frame;

the forced intra-frame area is an area other than the inter-frame area in the refreshed area of the current frame.

In a possible implementation manner, the decoding method based on the adaptive intra refresh mechanism, where the decoding method based on the forced intra region location and the inter region location decodes the code stream of the current frame, and the decoding method includes:

and if the current image block is determined to be the image block in the inter-frame area based on the area position of the inter-frame area, and the motion vector MV information of the current image block points to the refreshed area in the reference frame of the current image block, decoding the current image block based on the MV information of the current image block.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, after receiving the code stream of the current frame, the method further includes:

if the reference frame of the current image block to be decoded in the current frame is an intra-frame refresh frame and the current image block does not meet the condition that the current frame is an intra-frame refresh frame, the current image block falls in an un-refreshed area and the current frame and the reference frame are in the same refresh period, adjusting the MV information of the current image block to the pointed area as the MV information of the refreshed area in the reference frame;

and decoding the current image block based on the adjusted MV information.

Based on the decoding method based on the adaptive intra refresh mechanism provided above, in a possible implementation manner, the adjusted MV information satisfies the following condition:

MvE->x’＝Min(MvE->x,(RefAirBoundX–MaxFilterLineBuf–W–xE)<<PixelResolution)

MvE->y’＝Min(MvE->y,(RefAirBoundY–MaxFilterLineBuf–H–yE)<<PixelResolution)

MvE is a motion vector of the current image block, MvE- > X 'is a component of the motion vector of the adjusted current image block in the X direction, MvE- > y' is a component of the motion vector of the adjusted current image block in the y direction, MvE- > X is a component of the motion vector of the current image block carried in the code stream in the X direction, MvE- > y is a component of the motion vector of the current image block carried in the code stream in the y direction, RefAirBoundX and RefAirBoundY are virtual boundary coordinates AirBoundX and AirBoundY of the reference frame, MaxFilterLineBuf is a maximum line buffer of a filter used for loop filtering a virtual boundary of the reference frame, PixelResolution is a pixel precision of the motion vector, W is an image width of the current image block, H is an image height of the current image block, and (xE, yE) is a position of a left-upper-corner sample of the current image block in a luminance sample matrix of the current frame.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, the code stream carries loop filtering flag information, and the loop filtering flag information indicates to close or allow loop filtering on the virtual boundary.

Based on the decoding method based on the adaptive intra-frame refresh mechanism provided above, in a possible implementation manner, in a case that the loop filtering flag information indicates that loop filtering on the virtual boundary is allowed, the method further includes:

and for the region to be filtered determined based on the virtual boundary, not performing loop filtering operation on the region to be filtered.

Based on the decoding method based on the adaptive intra refresh mechanism provided above, in a possible implementation manner, in a case that the loop filter flag information indicates that loop filtering on the virtual boundary is allowed, the method further includes:

and for the area to be filtered determined based on the virtual boundary, performing loop filtering operation on the area to be filtered based on the pixels in the refreshed area.

and for the region to be filtered determined based on the virtual boundary, performing loop filtering operation on the region to be filtered based on the pixels in the refreshed region and the pixels in the non-refreshed region, wherein the region to be filtered located in the refreshed region cannot be used as the region pointed by the MV information of the image block in the next frame of the current frame in the decoding order.

Based on the foregoing decoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, in a scene of random access decoding, if the current frame is in a refresh cycle where an access point is located and the current frame includes an un-refreshed region, decoding a code stream of the current frame based on a region position of the refreshed region and a region position of the un-refreshed region, including:

analyzing a code stream of a current Largest Coding Unit (LCU) to obtain position information of the current LCU;

if the current LCU is determined to be located in the non-refreshed area based on the position information of the current LCU, the operation of reconstructing the current LCU is not executed, and the operation of analyzing the code stream of the next LCU of the current LCU is executed;

performing an operation to reconstruct the current LCU if the current LCU is located in the refreshed area.

Based on the foregoing provided decoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, in a scene of random access decoding, if the current frame is in a refresh cycle in which an access point is located and the current frame contains an un-refreshed region, decoding a code stream of the current frame based on a region position of the refreshed region and a region position of the un-refreshed region, including:

if the code stream start marking information of the current image slice indicates that the current image slice is located in the non-refreshed area, the operation of analyzing the code stream of the current image slice is not executed, and the initial position of the code stream of the next image slice is obtained in the code stream of the current frame based on the code stream start marking information so as to decode the next image slice;

and if the code stream start mark information of the current image slice indicates that the current image slice is located in the refreshed area, analyzing the code stream of the current image slice to decode the current image slice.

and decoding all code streams in the current frame.

In another aspect, an encoding method based on an adaptive intra refresh mechanism is provided, where the method includes:

transmitting a code stream of the current frame;

the code stream of the current frame carries frame type mark information, the frame type mark information indicates whether the current frame is an intra-frame refresh frame, under the condition that the frame type mark information indicates that the current frame is the intra-frame refresh frame, the code stream also carries virtual boundary position mark information, the virtual boundary position mark information indicates the position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed area from an un-refreshed area in the current frame.

Based on the above provided encoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, the virtual boundary position mark information includes first virtual boundary position mark information and second virtual boundary position mark information, the first virtual boundary position mark information indicates a position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information indicates a position of the virtual boundary in the vertical direction of the current frame;

wherein a position of the virtual boundary in the current frame in a vertical direction of the current frame is different from a position of a virtual boundary between a refreshed region and a non-refreshed region in an adjacent frame in the vertical direction of the adjacent frame, and the adjacent frame is a previous frame or a next frame of the current frame in a coding order.

Based on the above-mentioned encoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, the first virtual boundary position mark information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of the virtual boundary in the vertical direction of the current frame.

Based on the above provided encoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, the first virtual boundary position marker information is an image abscissa of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position marker information is an image ordinate of the virtual boundary in the vertical direction of the current frame;

the refreshed area is a rectangular area with the coordinates (0, 0) at the upper left corner of the image as the origin, the width as the abscissa of the image, and the height as the ordinate of the image, and the non-refreshed area is an area other than the refreshed area in the current frame.

Based on the above-mentioned encoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in the horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in the vertical direction of the current frame and a height of the current frame image.

Based on the above provided encoding method based on the adaptive intra refresh mechanism, in a possible implementation manner, the width precision of the refreshed region is the width of one coding unit CU, and the height precision of the refreshed region is the height of the CU; alternatively, the first and second electrodes may be,

Based on the coding method based on the adaptive intra-frame refreshing mechanism, in a possible implementation manner, the refreshed region includes a forced intra-frame region and an inter-frame region, and motion vector MV information of an image block in the inter-frame region carried in the code stream points to the refreshed region in a reference frame of the image block.

Based on the above provided encoding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, the refreshed region includes a forced intra-frame region and an inter-frame region, and the motion vector MV information of the image block in the inter-frame region carried in the code stream points to the refreshed region in the reference frame of the image block and/or the non-refreshed region in the reference frame of the image block.

Based on the coding method based on the adaptive intra-frame refresh mechanism, in a possible implementation manner, the code stream carries loop filtering flag information, and the loop filtering flag information indicates to close or allow loop filtering on the virtual boundary.

In another aspect, a decoding apparatus is provided, where the decoding apparatus includes:

the receiving module is used for receiving a code stream of a current frame, wherein the code stream of the current frame carries frame type mark information, and the frame type mark information indicates whether the current frame is an intra-frame refresh frame;

a processing module, configured to obtain virtual boundary position flag information carried in the code stream when the frame type flag information indicates that the current frame is an intra-frame refresh frame, where the virtual boundary position flag information indicates a position of a virtual boundary, and the virtual boundary is at least used to distinguish a refreshed region from an un-refreshed region in the current frame;

the processing module is further configured to determine, based on the virtual boundary position marker information, an area position of the refreshed area and an area position of the non-refreshed area;

the processing module is further configured to decode the code stream of the current frame based on the area position of the refreshed area and the area position of the non-refreshed area.

Based on the decoding apparatus provided above, in a possible implementation manner, the virtual boundary position flag information includes first virtual boundary position flag information and second virtual boundary position flag information, where the first virtual boundary position flag information indicates a position of the virtual boundary in the current frame horizontal direction, and the second virtual boundary position flag information indicates a position of the virtual boundary in the current frame vertical direction;

Based on the decoding apparatus provided above, in a possible implementation manner, the first virtual boundary position mark information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of the virtual boundary in the vertical direction of the current frame.

Based on the decoding apparatus provided above, in a possible implementation manner, the first virtual boundary position mark information is an image abscissa of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is an image ordinate of the virtual boundary in the vertical direction of the current frame;

Based on the decoding device provided above, in a possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in the horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in the vertical direction of the current frame and a height of the current frame image.

Based on the decoding device provided above, in a possible implementation manner, the width precision of the refreshed region is the width of one coding unit CU, and the height precision of the refreshed region is the height of the CU; alternatively, the first and second electrodes may be,

Based on the decoding device provided above, in a possible implementation manner, the refreshed region includes a forced intra region and an inter region;

the processing module is used for:

Based on the decoding device provided above, in a possible implementation manner, the inter-frame region is a refreshed region of the previous frame;

Based on the decoding apparatus provided above, in a possible implementation manner, the processing module is configured to:

Based on the decoding apparatus provided above, in a possible implementation manner, the processing module is further configured to:

and decoding the current image block based on the adjusted MV information.

Based on the decoding apparatus provided above, in a possible implementation manner, the adjusted MV information satisfies the following condition:

Based on the decoding device provided above, in a possible implementation manner, the code stream carries loop filtering flag information, where the loop filtering flag information indicates to close or allow loop filtering on the virtual boundary.

Based on the decoding apparatus provided above, in a possible implementation manner, in a case that the loop filtering flag information indicates that loop filtering on the virtual boundary is allowed, the processing module is further configured to:

Based on the decoding apparatus provided above, in a possible implementation manner, in a scenario of random access decoding, if the current frame is in a refresh period where an access point is located and the current frame contains an un-refreshed area, the processing module is configured to:

analyzing a code stream of a current maximum coding unit (LCU) to obtain position information of the current LCU;

and decoding all code streams in the current frame.

In another aspect, an encoding apparatus is provided, the encoding apparatus including:

the sending module is used for sending the code stream of the current frame;

Based on the coding device provided above, in a possible implementation manner, the virtual boundary position mark information includes first virtual boundary position mark information and second virtual boundary position mark information, where the first virtual boundary position mark information indicates a position of the virtual boundary in the current frame horizontal direction, and the second virtual boundary position mark information indicates a position of the virtual boundary in the current frame vertical direction;

Based on the encoding apparatus provided above, in a possible implementation manner, the first virtual boundary position mark information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of the virtual boundary in the vertical direction of the current frame.

Based on the encoding apparatus provided above, in a possible implementation manner, the first virtual boundary position mark information is an image abscissa of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is an image ordinate of the virtual boundary in the vertical direction of the current frame;

Based on the encoding apparatus provided above, in a possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in the horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in the vertical direction of the current frame and a height of the current frame image.

Based on the coding apparatus provided above, in a possible implementation manner, the width precision of the refreshed region is the width of one coding unit CU, and the height precision of the refreshed region is the height of the CU; alternatively, the first and second electrodes may be,

Based on the encoding device provided above, in a possible implementation manner, the refreshed area includes a forced intra-frame area and an inter-frame area, and the motion vector MV information of the image block in the inter-frame area carried in the code stream points to the refreshed area in the reference frame of the image block.

Based on the encoding device provided above, in a possible implementation manner, the refreshed area includes a forced intra-frame area and an inter-frame area, and the motion vector MV information of the image block in the inter-frame area carried in the code stream points to the refreshed area in the reference frame of the image block and/or the non-refreshed area in the reference frame of the image block.

Based on the coding device provided above, in a possible implementation manner, the code stream carries loop filtering flag information, where the loop filtering flag information indicates to close or allow loop filtering on the virtual boundary.

In another aspect, a decoding-end device is provided, where the decoding-end device includes:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform any one of the above-mentioned decoding methods based on the adaptive intra refresh mechanism.

In another aspect, an encoding end device is provided, where the encoding end device includes:

a processor;

a memory for storing processor-executable instructions;

In another aspect, a computer-readable storage medium is provided, which has instructions stored thereon, and when executed by a processor, the instructions implement any one of the above decoding method based on the adaptive intra refresh mechanism or the decoding method based on the adaptive intra refresh mechanism.

In another aspect, a computer program product is provided, which comprises instructions that, when run on a computer, cause the computer to perform any of the above-described decoding method based on an adaptive intra refresh mechanism or decoding method based on an adaptive intra refresh mechanism.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

the method comprises the steps of adding virtual boundary position mark information in a code stream of a current frame, wherein the virtual boundary position mark information can indicate the position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed region from an un-refreshed region in the current frame, so that when a coding end divides a forced intra-frame region of the current frame, the virtual boundary can be set arbitrarily based on actual requirements, and then the position of the virtual boundary is indicated through the virtual boundary position mark information, so that the flexibility of dividing the forced intra-frame region is improved, namely, the embodiment of the application provides an adaptive intra-frame refreshing mechanism for adaptively dividing the forced intra-frame region. For example, the encoding end may divide the forced intra-frame region of the current frame based on the boundary of the object included in the image content of the current frame, so that the virtual boundary between the refreshed region and the non-refreshed region may coincide with the boundary of the object in the image content of the current frame as much as possible.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic diagram of an intra refresh mechanism according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an image sequence after an intra refresh mechanism is adopted according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an image sequence after an intra refresh mechanism is adopted according to an embodiment of the present application;

fig. 4 is a flowchart of an encoding method based on an adaptive intra refresh mechanism according to an embodiment of the present application;

fig. 5 is a schematic diagram illustrating a distribution of regions of an intra refresh frame in an adaptive intra refresh mechanism according to an embodiment of the present application;

fig. 6 is a flowchart of a decoding method based on an adaptive intra refresh mechanism according to an embodiment of the present application;

FIG. 7 is a diagram of a reference frame provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of a decoding apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an encoding apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a terminal according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.

Before explaining the method provided by the embodiment of the present application, an application scenario related to the embodiment of the present application is explained first.

In video coding and decoding, an image frame used for reference is called an I frame, and the I frame is also called a key frame. When the I frame is coded, the compression rate is very low, and the purpose is that the I frame can be obtained by directly decoding the code stream based on the I frame during subsequent decoding, so that other image frames do not need to be referred. During encoding, an image frame which is encoded based on a difference value between a previous image frame and a current image frame is called a P frame, and a code stream of the encoded P frame carries the difference value between the previous image frame and the current image frame, so that the current image frame is obtained by decoding the difference value between the previous image frame and the code stream which are obtained by decoding based on the I frame. In addition, during encoding, an image frame which is encoded based on a difference value between a previous frame image and a current image frame and a difference value between the current image frame and a next frame image is called a B frame, and a code stream of the encoded B frame carries the difference value between the previous frame image and the current image frame and the difference value between the current image frame and the next frame image, so that the current image frame is obtained by decoding based on the I frame, the decoded previous frame image and the decoded next frame image and the difference value in the code stream. A sequence of an encoded I frame and B frames or P frames is referred to as an image sequence. This image sequence is also referred to as IPPP or IBBB sequence.

When a decoding end accesses an IPPP or IBBB sequence in a random access mode, the code rate of an I frame is usually far greater than that of a P frame or a B frame, so that the decoding time of the I frame is longer than that of the P frame or the B frame. Especially in weak network environments, the time required to decode an I-frame is longer, resulting in the possibility of video seizure.

To avoid video sticking due to the long time required to decode an I-frame, an intra refresh mechanism is currently provided in the art. The core idea of the intra-frame refresh mechanism is as follows: the code rate of the I frame is dispersed to a plurality of P frames, the coding mode of a small block area of each P frame is a forced intra-frame mode, the coding modes of other areas are allowed to adopt an inter-frame mode, and finally the code rate of each P frame is much smaller than that of the original I frame. Meanwhile, the forced intra-frame areas of different P frames are required to be mutually intersected in the intra-frame refreshing mechanism, so that after a plurality of P frames, the whole image area can be refreshed based on the forced intra-frame mode.

For convenience of the following description, the principle of the intra refresh mechanism is described in detail below.

Fig. 1 is a schematic diagram of an intra refresh mechanism according to an embodiment of the present disclosure. The intra refresh mechanism in fig. 1 is to spread the code rate of I-frames over four P-frames, each P-frame having a region that is a mandatory intra region. Where the region labeled (r) is a forced intra region. The region marked c is a refreshed region of the previous frame in coding order, the coding mode of which allows the use of an inter mode, and is therefore also referred to as the inter region of the current frame. The area marked (c) is an unwritten area whose coding mode allows the use of the inter mode.

As shown in fig. 1, the complete image frame is divided into 4 forced intra-frame areas, and the refresh period includes 4 intra-frame refresh frames, which are 4 frame images from left to right in fig. 1. For the first frame image in the refresh period, the image comprises an intra-frame refresh area and an un-refresh area, and the refreshed area in the first frame image only comprises the forced intra-frame area in the current image frame because the first frame image is the first frame image in the refresh period. And for a second frame image in the refresh period, the image comprises a refreshed area, an intra-frame refreshed area and an un-refreshed area in the first frame image, and the refreshed area of the first frame image in the second frame image and the intra-frame refreshed area in the current frame image are combined and then are collectively referred to as the refreshed area of the second frame image. And for a third frame image in the refresh period, the image comprises a refreshed area, an intra-frame refreshed area and an un-refreshed area in the second frame image, and the refreshed area of the second frame image in the third frame image and the intra-frame refreshed area in the current frame image are combined and then collectively referred to as the refreshed area of the third frame image. And for a fourth frame image in the refresh period, the image comprises a refreshed area in the third frame image and an intra-frame refreshing area, and the refreshed area in the third frame image in the fourth frame image and the intra-frame refreshing area in the current image frame are combined and then collectively referred to as the refreshed area of the fourth frame image. Since the complete image was previously divided into 4 forced intra areas, there are no non-refreshed areas in the fourth frame image.

In addition, the non-refreshed area is also called a dirty (dirty) area, and the refreshed area of the previous frame and the forced intra-frame area of the current frame in the coding order may be combined to be collectively called a clean (clean) area of the current frame, and the boundary between the two areas is the boundary in fig. 1. The virtual boundary according to the embodiment of the present application is also a boundary for distinguishing a clean area from a dirty area, and only the dividing manner of the boundary is more flexible, and the virtual boundary may be a broken line boundary and is not limited to a vertical boundary. In the embodiment of the present application, for any intra-frame refresh frame, the virtual boundary between the refreshed area and the non-refreshed area of the intra-frame refresh frame refers to a boundary between the refreshed area and the non-refreshed area, and the boundary is also a boundary between the clean area and the dirty area of the intra-frame refresh frame.

Fig. 2 is a schematic diagram of an image sequence after an intra refresh mechanism is adopted according to an embodiment of the present application. As shown in fig. 2, four frames I2(I frame), P8(P frame), P9(P frame), and P10(P frame) in the original image sequence (referred to as the original sequence in fig. 2) are replaced by four intra-frame refresh frames, which are labeled as X1, X2, X3, and X4, resulting in an image sequence based on the intra-frame refresh mechanism (referred to as the intra-frame refresh sequence in fig. 2). All four intra-frame refresh frames have an area which is a forced intra-frame area, and the coding modes of other areas allow the adoption of an inter-frame mode, so that the four intra-frame refresh frames can be regarded as four 'large P frames', and the code rate is approximately 1/4 of the code rate of the original I2 frame. Further, these four intra refresh frames may also be referred to simply as IR frames.

In the image sequence shown in fig. 2, in the random access mode, assuming that the I2 frame in the original image sequence is a random access point, the frames following the I2 frame can be decoded independently, so that the P8 frame can only refer to the I2 frame, and cannot refer to the P7 frame. However, in the image sequence based on the intra refresh mechanism (referred to as intra refresh sequence in fig. 2), the X1 frame is a random access point, and the inter-coded areas in the non-refreshed areas in the X1 frame can refer to the P7 frame but may not be decoded correctly (because the P7 frame may not be received yet at the time of random access). Similarly, the non-refreshed regions of the X2 frame and the X3 frame may not be decoded correctly. Until all the frames from X1 to X4 are refreshed, at this time, because the inter-frame region in the refreshed region in the X4 frame can be obtained by decoding the code stream of the forced intra-frame region from X1 to X3, and the forced intra-frame region in the refreshed region in the X4 frame can be obtained by decoding in the forced intra-frame mode, the X4 frame without the un-refreshed region can be completely and correctly decoded. Therefore, the frames X1, X2, X3, and X4 in fig. 2 are a refresh cycle, and only the last frame in the refresh cycle can be decoded correctly and completely, so that the frame is called a Recovery Point (Recovery Point).

It should be noted that, in the case of random access, only the recovery point and the image frame after the recovery point are displayed to the user, and other images in the refresh cycle where the random access point is located are not displayed because they may not be decoded correctly. In the non-random access situation, all image frames can be completely and correctly decoded, so that all image frames can be displayed to the user.

In addition, in order to ensure the code stream decoding function after being encoded based on the intra-frame refreshing mechanism, the current intra-frame refreshing mechanism limits the following conditions for the intra-frame mode/inter-frame mode and the loop filtering in the decoding mode:

the first condition is as follows: for any intra-frame refresh frame, the block of the refreshed region on the intra-frame refresh frame can only refer to the refreshed regions of other intra-frame refresh frames in the same refresh period for decoding, and cannot refer to the non-refreshed regions of other intra-frame refresh frames for decoding; the co-located block of the current block in the reference frame in the TMVP (temporal motion vector prediction) mechanism cannot fall into an un-refreshed area of the reference frame, where the co-located block refers to an image block in the reference frame that is located at the same position as the current block, that is, an area pointed by MV information of the image block of the current image frame cannot fall into the un-refreshed area of the reference frame. It should be noted that, in the embodiment of the present application, the current block and the current image block are the same concept, and for convenience of description, the current image block may be simply referred to as the current block.

And a second condition: for any intra-refresh frame, there is no reference limit when decoding the blocks of the non-refreshed area on the intra-refresh frame.

And (3) carrying out a third condition: for an image frame which is not an intra-frame refresh frame in an image sequence, a block of the image frame allows for decoding by referring to a refreshed area of the intra-frame refresh frame in a random access period, but cannot decode by referring to an unswept area of the intra-frame refresh frame in the random access period; the co-located block of the current block in the TMVP mechanism in the reference frame cannot fall into the non-refreshed area of the reference frame.

And a fourth condition: for any intra-frame refresh frame, the loop filtering cannot cross the boundary between the refreshed area and the non-refreshed area of the intra-frame refresh frame, that is, the loop filtering operation is not performed on the virtual boundary between the refreshed area and the non-refreshed area.

At present, a GDR (Gradual Decoding Refresh) technology in a VVC (universal Video Coding) standard provides an intra-frame Refresh mechanism. The specific contents of the intra refresh mechanism are as follows.

The refreshed area and the non-refreshed area are first divided by a vertical division manner. In the vertical division, a virtual boundary between the refreshed area and the non-refreshed area is a boundary line parallel to the Y axis in the image coordinate system. The partition boundary is aligned to the width and height of the minimum CU (coding unit) (the minimum of the CU is 8 pixels), that is, the width of the refreshed area is an integral multiple of the width of the minimum CU, and the height of the refreshed area is the image height of the intra refresh frame.

Secondly, forced inter-frame areas on different intra-frame refresh frames are divided equidistantly, and the width of the forced inter-frame area is equal to the total width/refresh period of the image frame. The refresh period refers to the number of intra refresh frames required for correctly decoding a complete image frame, for example, the refresh period in fig. 3 is 6. For example, the encoding end may set the refresh period based on a header parameter recovery _ poc _ cnt of a code stream of an intra-frame refresh frame, and for the image sequence shown in fig. 3, may set recovery _ poc _ cnt to 6, which indicates that the refresh period in which the intra-frame refresh frame is located is 6. In addition, the refreshing sequence is from left to right as a default, the refreshing sequence does not need to be explicitly indicated in the code stream, and the decoding end decodes each intra-frame refreshing frame according to the refreshing sequence from left to right as a default.

In addition, for any image frame, the encoding end may also identify whether the current image frame is an intra-frame refresh frame based on a picture header parameter gdr _ pic _ flag of a code stream of the image frame. For example, when gdr _ pic _ flag is 1, it indicates that the current image frame is an intra-frame refresh frame, and when gdr _ pic _ flag is 0, it indicates that the current image frame is not an intra-frame refresh frame.

In addition, the intra refresh mechanism in the VCC standard also includes the four constraints of condition one to condition 4, which are not described herein again.

The forced intra-frame area in the intra-frame refresh mechanism in the VCC standard is not flexible enough in dividing manner, and only can be equidistant vertical strip areas, and the area cannot be adaptively divided according to the image content. Furthermore, since the intra refresh mechanism in the VCC standard does not allow loop filtering across the virtual boundary between the refreshed and non-refreshed regions, the image quality on the virtual boundary is poor.

Based on the problems occurring in the intra-frame refresh mechanism in the VCC standard, the embodiment of the present application provides a coding and decoding method based on an adaptive intra-frame refresh mechanism, in which a more flexible method for partitioning an intra-frame region is provided, and a partition mode coding end is configurable. In addition, the virtual boundary between the refreshed area and the non-refreshed area can be enabled to coincide with the object boundary in the image as much as possible, so that the influence on the decoded image quality due to the fact that loop filtering cannot be performed on the virtual boundary can be avoided.

The method provided by the embodiment of the present application can adaptively divide the virtual boundary between the refreshed area and the non-refreshed area according to the image content, so the intra-frame refresh frame related to the embodiment of the present application can also be referred to as an adaptive intra-frame refresh frame, and for convenience of description, the adaptive intra-frame refresh frame related to the embodiment of the present application is referred to as an intra-frame refresh frame or a refresh frame.

The following explains the coding and decoding method based on the adaptive intra refresh mechanism provided in the embodiment of the present application in detail.

Fig. 4 is an encoding method based on an adaptive intra refresh mechanism according to an embodiment of the present application. As shown in fig. 4, the method includes the following steps.

Step 401: the method comprises the steps that a coding end codes a code stream of a current frame, wherein the code stream of the current frame carries frame type mark information, the frame type mark information indicates whether the current frame is an intra-frame refreshing frame, under the condition that the frame type mark information indicates that the current frame is the intra-frame refreshing frame, the code stream also carries virtual boundary position mark information, the virtual boundary position mark information indicates the position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed area from an un-refreshed area.

The virtual boundary may be a boundary line of a refreshed area in the current frame, a boundary line of an un-refreshed area in the current frame, or a divided boundary line of a refreshed area and an un-refreshed area. The examples of these three virtual boundaries are essentially the same.

The following embodiments will be described by taking as an example a boundary between a refreshed area and an un-refreshed area, which are divided by a virtual boundary. That is, the virtual boundary position flag information may indicate a position of a virtual boundary between the refreshed region and the non-refreshed region in the current frame, and the position of the virtual boundary may specifically include a position of the virtual boundary in the horizontal direction of the current frame and a position of the virtual boundary between the refreshed region and the non-refreshed region in the current frame in the vertical direction of the current frame.

However, it should be noted that the virtual boundary in the adaptive intra-frame refresh scheme provided in the embodiment of the present application is not limited to the above explanation, and the virtual boundary may be any one of the three virtual boundary examples described above. In the embodiment of the application, the virtual boundary position mark information is newly added in the code stream of the current frame, and the virtual boundary position mark information can indicate the position of the virtual boundary between the refreshed region and the non-refreshed region in the current frame in the horizontal direction of the current frame and the position of the virtual boundary between the refreshed region and the non-refreshed region in the current frame in the vertical direction of the current frame, so that when the coding end divides the forced intra-frame region of the current frame, the virtual boundary can be arbitrarily set based on actual requirements, and then the virtual boundary is indicated by the virtual boundary position mark information, so that the flexibility of dividing the forced intra-frame region is improved.

For example, the encoding end may divide the forced intra-frame region of the current frame based on the boundary of the object included in the image content of the current frame, so that the virtual boundary between the refreshed region and the non-refreshed region may coincide with the boundary of the object in the image content of the current frame as much as possible.

Optionally, the encoding end may also divide the forced intra-frame region of the current frame based on other requirements, which is not illustrated herein.

The frame type flag information indicates whether the current frame is an intra refresh frame. For example, the encoding end may use a picture header parameter air _ pic _ flag in the code stream of the current frame as the frame type flag information. That is, whether the current frame is an intra refresh frame (IR frame) is identified by the picture header parameter air _ pic _ flag. For example, when air _ pic _ flag is 1, it indicates that the current frame is an intra-frame refresh frame, and when air _ pic _ flag is 0, it indicates that the current frame is not an intra-frame refresh frame.

The following examples illustrate several alternative embodiments of the virtual boundary position marker information provided by embodiments of the present application. It should be noted that the following several embodiments of the virtual boundary position mark information are only used for illustration, and do not limit the function of the virtual boundary position mark information provided in the embodiments of the present application, and any mark information capable of indicating the position of the virtual boundary between the refreshed area and the non-refreshed area in the current frame in the horizontal direction of the current frame and the position of the virtual boundary between the refreshed area and the non-refreshed area in the current frame in the vertical direction of the current frame is within the range of the virtual boundary position mark information provided in the embodiments of the present application.

Example virtual boundary position marker information one:

in example one, the virtual boundary position marker information includes first virtual boundary position marker information indicating a position of a virtual boundary between the refreshed region and the non-refreshed region in a current frame horizontal direction, and second virtual boundary position marker information indicating a position of a virtual boundary between the refreshed region and the non-refreshed region in a current frame vertical direction.

Wherein, the position of the virtual boundary between the refreshed area and the non-refreshed area in the current frame in the vertical direction of the current frame is different from the position of the virtual boundary between the refreshed area and the non-refreshed area in the adjacent frame in the vertical direction of the adjacent frame, and the adjacent frame is the previous frame or the next frame of the current frame in the coding sequence. That is, in the same refresh period, the positions of virtual boundaries between the refreshed areas and the non-refreshed areas of the refreshed frames in adjacent two frames in the vertical direction of the corresponding frames are different. Therefore, the forced intra-frame area obtained by division can be prevented from being a vertical strip-shaped area, and meanwhile, the position of the virtual boundary between the refreshed area and the non-refreshed area in the horizontal direction of the current frame and the position of the virtual boundary between the refreshed area and the non-refreshed area in the vertical direction of the current frame can be randomly configured by the encoding end, so that the widths of the forced intra-frame areas of the intra-frame refreshing frames are not required to be the same. Therefore, the coding end can realize the unequal distance division of the forced intra-frame areas of different intra-frame refreshing frames in the same refreshing period through the virtual boundary position mark information.

For example, the picture header parameters bd _ x and bd _ y are added to the code stream of the current frame. bd _ x is the first virtual boundary position mark information, and bd _ y is the second virtual boundary position mark information. The virtual boundary between the refreshed area and the non-refreshed area in the current frame can be identified by bd _ x and bd _ y.

In the first example of the virtual boundary position mark information, in particular, in one possible implementation manner, the first virtual boundary position mark information is a pixel position of a virtual boundary between the refreshed region and the non-refreshed region in the horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of a virtual boundary between the refreshed region and the non-refreshed region in the vertical direction of the current frame.

For example, the first virtual boundary position mark information is an image abscissa of a virtual boundary between the refreshed region and the non-refreshed region in the horizontal direction of the current frame, and the second virtual boundary position mark information is an image ordinate of a virtual boundary between the refreshed region and the non-refreshed region in the vertical direction of the current frame. In this case, the refreshed area of the current frame is a rectangular area with the upper left-hand coordinates (0, 0) of the image as the origin, the width as the horizontal coordinate of the image, and the height as the vertical coordinate of the image, and the non-refreshed area of the current frame is an area other than the refreshed area of the current frame.

Fig. 5 is a schematic diagram illustrating a distribution of regions of each intra refresh frame in an adaptive intra refresh mechanism according to an embodiment of the present application. As in fig. 1, in fig. 5, the region marked with (r) is a forced intra region. The region marked (c) is a refreshed region of the previous frame in coding order, the coding mode of which allows to use inter mode, hence also called inter region of the current frame. The area marked- (c) is the non-refreshed area of the current frame.

As shown in fig. 5, the refreshed area of any intra-frame refresh frame in fig. 5 is a rectangular area with coordinates (0, 0) as the upper left corner, a width as the image abscissa bd _ x, and a height as the image ordinate bd _ y, and the non-refreshed area of the intra-frame refresh frame (c) is the other area except the refreshed area in the intra-frame refresh frame. In the first frame image from left to right in fig. 5, the refreshed area is the forced intra-frame area (i). In the second frame image to the fourth frame image, the refreshed area comprises a forced intra-frame area (I) and an inter-frame area (II), wherein the inter-frame area (II) in any frame image is the refreshed area of the previous frame image.

In one example of the virtual boundary position flag information, in particular, in another possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in a horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in a vertical direction of the current frame and a height of the current frame image.

For example, the first virtual boundary position flag information bd _ x is a ratio between a width of the refreshed region in the horizontal direction of the current frame and a width of the current frame image, and the second virtual boundary position flag information bd _ y is a ratio between a height of the refreshed region in the vertical direction of the current frame and a height of the current frame image. Wherein bd _ x or bd _ y may be 1/4 or 3/5, etc.

The following takes tables 1 to 3 as examples to describe specific implementations of various syntax elements in the first example of virtual boundary position marker information.

TABLE 1

Wherein, air _ enable _ flag is an adaptive intra refresh allowed flag. The adaptive intra refresh allowed flag is a binary variable. The value of air _ enable _ flag is '1' indicating that the adaptive intra refresh techniques provided by embodiments of the present application can be used in image sequences. The value of air _ enable _ flag is '0' to indicate that the adaptive intra refresh technique provided by the embodiments of the present application cannot be used. The value of AirEnableFlag in the subsequent embodiments is equal to the value of air _ enable _ flag. If air _ enable _ flag does not exist in the code stream, the value of the AirEnablFlag is 0.

TABLE 2

TABLE 3

In tables 2 and 3, picture _ air _ pic _ flag is a picture-level adaptive intra refresh frame flag. The image-level adaptive intra refresh frame flag is a binary variable. A value of picture _ air _ pic _ flag of '1' indicates that the current frame is an intra refresh frame; a value of '0' for picture _ air _ pic _ flag indicates that the current frame is not an intra refresh frame. The value of PictureAirPicFlag referred to in the subsequent embodiments is equal to the value of picture _ air _ pic _ flag. If picture _ air _ pic _ flag is not present in the codestream, the value of PictureAirPicFlag is 0.

In the case where the value of picture _ air _ pic _ flag is '1', the current frame is an intra-frame refresh frame, and the image abscissa of the virtual boundary between the refreshed area and the non-refreshed area of the current frame is air _ bound _ x, and the image ordinate of the virtual boundary between the refreshed area and the non-refreshed area of the current frame is air _ bound _ y, where the virtual boundary is the boundary between the refreshed area and the non-refreshed area. The refreshed area is a rectangular area with (0, 0) as the top left corner, the width of air _ bound _ x and the height of air _ bound _ y, and the un-refreshed area is the remaining image area in the current frame. The values of airbound x and airbound y in the subsequent embodiments are equal to the values of air _ bound _ x and air _ bound _ y, respectively. If the code stream does not have the air _ bound _ x and the air _ bound _ y, the values of the air bound X and the air bound Y are both 0.

It should be noted that, table 1 to table 3 are only optional examples of syntax elements provided in the embodiment of the present application, and the embodiment of the present application does not limit the concrete expression form of these syntax elements.

In addition, in order to increase the subsequent decoding speed, the encoding side should also consider the boundary accuracy of the virtual boundary between the refreshed region and the non-refreshed region when dividing the region into the refreshed region and the non-refreshed region. That is, the width accuracy and the height accuracy of the refreshed area are considered. Here, the width accuracy of the refreshed area refers to the minimum width of the refreshed area, and the actual width of the refreshed area is usually configured to be an integral multiple of the width accuracy. The height accuracy of the refreshed area refers to the minimum height of the refreshed area, and the actual height of the refreshed area is usually configured as an integral multiple of the height accuracy.

Several alternative embodiments of boundary accuracy provided by embodiments of the present application are illustrated below. It should be noted that the following several boundary accuracy embodiments are only used for illustration, and do not limit the scope of the boundary accuracy provided by the embodiments of the present application.

Boundary accuracy example one:

the width precision of the refreshed area is the width of one coding unit CU, and the height precision of the refreshed area is the height of the CU.

For example, the minimum CU width and height are 8 pixels, and then bd _ x and bd _ y may be multiples of 8 in the case where bd _ x is the image abscissa of the virtual boundary between the refreshed area and the non-refreshed area in the horizontal direction of the current frame, and bd _ y is the image ordinate of the virtual boundary between the refreshed area and the non-refreshed area in the vertical direction of the current frame.

Boundary accuracy example two:

the width precision of the refreshed area is the width of one LCU (largest coding unit), and the height precision of the refreshed area is the height of the LCU.

For example, an LCU has a width and height of 128 pixels, where bd _ x is the horizontal coordinate of the image of the virtual boundary between the refreshed area and the non-refreshed area in the horizontal direction of the current frame, and bd _ y is the vertical coordinate of the image of the virtual boundary between the refreshed area and the non-refreshed area in the vertical direction of the current frame, bd _ x and bd _ y may be multiples of 128, and the virtual boundary of the refreshed area is aligned to the boundary of the LCU.

Boundary accuracy example three:

the width precision of the refreshed area is the width of one image patch (patch), and the height precision of the refreshed area is the height of the image patch, which comprises one or more LCUs. Wherein an image slice is an area of one or more LCUs.

In addition, in order to further improve the compression rate of the current frame, in inter-frame coding, the relative displacement between the current coding block and the best matching block in its reference image frame is represented by MV motion vector (motion vector) information. Each divided block has corresponding MV information to be transmitted to a decoding side so that the decoding side decodes the current block based on the MV information of the respective blocks. The image block in the reference image frame according to which the MV information of the current block in the current frame is obtained is called an image block pointed by the MV information of the current block, and the pointed image block is also called a co-located image block or a reference block of the current block. That is, when the current frame is encoded, the encoded code stream also carries the MV information of the current block.

The aforementioned reference image frame and the reference frame are the same concept, and the embodiment of the present application does not limit this.

In the embodiment of the present application, when performing inter-frame coding, the encoding end may be implemented by any one of the following embodiments. It should be noted that the following several inter-frame coding embodiments are only used for illustration, and do not limit the scope of the inter-frame coding modes provided by the embodiments of the present application.

Inter-frame coding example one:

the refreshed area of the current frame includes a forced intra area and an inter area. At this time, the MV information of the image block in the inter-frame region carried in the code stream of the current frame points to the refreshed region in the reference frame of the corresponding image block.

That is, the MV information of the image block is limited at the encoding end, and it is required that the reference block in the reference frame pointed by the MV information of the image block cannot fall into the non-refreshed area of the reference frame. In such a scenario, if the current block is located in the inter-frame region, the MV information of the current block that is taken by the decoding end by default is always a refreshed region of the reference frame pointing to the current block, and the decoding end can directly decode the current block based on the MV information without any adjustment.

Inter-frame coding example two:

the refreshed area of the current frame includes a forced intra area and an inter area. At this time, the MV information of the image block in the inter-frame region carried in the code stream of the current frame points to the refreshed region in the reference frame of the corresponding image block and/or the non-refreshed region in the reference frame of the corresponding image block.

That is, the MV information of the image block is not limited at the encoding end. In this scenario, if the current block is located in the inter-frame region, the MV of the current block taken by the decoding end does not necessarily point to the refreshed region of the reference frame of the current block, and at this time, the decoding end needs to adjust the MV information so as to perform decoding based on the adjusted MV information. Specific implementations for adjusting the MV information will be described in detail later.

In addition, in order to avoid poor image quality at the decoded virtual boundary, in the embodiment of the present application, loop filter flag information may be further carried in the code stream of the current frame, where the loop filter flag information indicates to turn off or allow loop filtering on the virtual boundary between the refreshed region and the non-refreshed region. Whether the decoding end needs to perform loop filtering in the area near the virtual boundary can be indicated by the loop filtering flag information, and the specific operation of the decoding end will be described in detail in the following embodiments, which will not be set forth herein.

The above examples of the virtual boundary position marker information, the boundary accuracy, and the inter-frame coding may be combined arbitrarily, and the present embodiment is not limited to this.

In addition, the refresh sequence in the embodiment of the present application is default from top left to bottom right, the refresh sequence does not need to be explicitly indicated in the code stream, and the decoding end defaults to decode each intra-frame refresh frame according to the refresh sequence from top left to bottom right.

Step 402: and the encoding end sends the code stream of the current frame to the decoding end.

After the encoding end completes the encoding in step 401, the decoding end may send the code stream of the current frame, so that the decoding end can decode the code stream to obtain the current frame.

In summary, the virtual boundary position flag information is newly added in the code stream of the current frame, and the virtual boundary position flag information can indicate the position of the virtual boundary, and the virtual boundary is at least used for distinguishing the refreshed region from the non-refreshed region in the current frame, so that when the coding end divides the forced intra-frame region of the current frame, the virtual boundary can be arbitrarily set based on actual requirements, and then the position of the virtual boundary is indicated by the virtual boundary position flag information, so that the flexibility of dividing the forced intra-frame region is improved, that is, the embodiment of the present application provides an adaptive intra-frame refresh mechanism for adaptively dividing the forced intra-frame region. For example, the encoding end may divide the forced intra-frame region of the current frame based on the boundary of the object included in the image content of the current frame, so that the virtual boundary between the refreshed region and the non-refreshed region may coincide with the boundary of the object in the image content of the current frame as much as possible.

The decoding method based on the intra refresh mechanism provided by the embodiment of the present application is explained in detail by the embodiment shown in fig. 6. As shown in fig. 6, the decoding method includes the following steps.

Step 601: and the decoding end receives the code stream of the current frame, wherein the code stream of the current frame carries frame type mark information, and the frame type mark information indicates whether the current frame is an intra-frame refresh frame.

For the related explanation of the frame type flag information, reference may be made to step 401 in the encoding-side embodiment, which is not described herein again.

Step 602: and under the condition that the frame type mark information indicates that the current frame is an intra-frame refreshing frame, the decoding end acquires virtual boundary position mark information carried in a code stream, wherein the virtual boundary position mark information indicates the position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed area from an un-refreshed area in the current frame.

For the related explanation of the virtual boundary position mark information, reference may also be made to step 401 in the encoding end embodiment, which is not described herein again.

Step 603: based on the virtual boundary position mark information, the decoding end determines the region position of the refreshed region and the region position of the non-refreshed region in the current frame.

Based on step 401 in the encoding end embodiment, the virtual boundary position flag information has different virtual boundary position flag information examples, and the implementation manner of determining the region position of the refreshed region and the region position of the non-refreshed region in the current frame by the decoding end is also different corresponding to the different virtual boundary position flag information examples. The details are described below.

An example of the virtual boundary position flag information in step 401 is one, at this time, the virtual boundary position flag information includes first virtual boundary position flag information indicating a position of a virtual boundary between the refreshed region and the non-refreshed region in the horizontal direction of the current frame, and second virtual boundary position flag information indicating a position of a virtual boundary between the refreshed region and the non-refreshed region in the vertical direction of the current frame. In this scenario, the implementation manner of step 603 is specifically: the area position of the refreshed area in the current frame is determined according to the first virtual boundary position mark information and the second virtual boundary position mark information, and then the area position of the non-refreshed area in the current frame can be obtained.

For example, bd _ x is the first virtual boundary position mark information, bd _ y is the second virtual boundary position mark information, bd _ x is an image abscissa of a virtual boundary between the refreshed region and the non-refreshed region in the horizontal direction of the current frame, and bd _ y is an image ordinate of a virtual boundary between the refreshed region and the non-refreshed region in the vertical direction of the current frame. At this time, the refreshed region of the previous frame determined by the decoding end is a rectangular region whose image coordinates are [0, bd _ x) × [0, bd _ y), and the remaining region excluding the rectangular region in the current frame is an un-refreshed region.

It should be noted that the above implementation manner is only an implementation manner of the decoding end with respect to the example of the virtual boundary position mark information in step 401, and does not limit a specific implementation manner of the decoding end determining the region position of the refreshed region and the region position of the non-refreshed region in the current frame, when using the embodiment of the present application, a specific implementation manner of the decoding end determining the region position of the refreshed region and the region position of the non-refreshed region in the current frame may be formulated according to a specific example of the virtual boundary position mark information, which is not illustrated herein one by one.

Step 604: and decoding the code stream of the current frame by the decoding end based on the area position of the refreshed area and the area position of the non-refreshed area.

Based on the foregoing, for other intra-frame refresh frames except for the first intra-frame refresh frame in the refresh period, the refreshed areas of these other intra-frame refresh frames include the mandatory intra-frame area and the inter-frame area, wherein the inter-frame area needs to be decoded by using the inter-frame decoding method, and the mandatory intra-frame area can only be decoded by using the intra-frame decoding method. Therefore, for these other intra-frame refresh frames, the region position of the forced intra-frame region and the region position of the inter-frame region in these other intra-frame refresh frames need to be further determined, so as to perform decoding based on different decoding manners.

Therefore, in a possible implementation manner, the implementation manner of step 604 is specifically: determining the region position of a forced intra region in the current frame and the region position of an inter region based on the region position of a refreshed region in the current frame and the refreshed region of a previous frame in the decoding order of the current frame; and decoding the code stream of the current frame based on the area position of the forced intra-frame area in the current frame and the area position of the inter-frame area.

Exemplarily, a first example is given to the virtual boundary position mark information in step 401, where the inter-frame region in the determined refreshed region of the current frame is the refreshed region of the previous frame; and determining the forced intra-frame area in the refreshed area of the current frame as an area except the inter-frame area in the refreshed area of the current frame.

That is, the region position of the forced intra region in the current frame and the region position of the inter region can be obtained by the following formulas:

the inter-frame area of the current frame is the refreshed area of the previous frame in the decoding sequence;

the forced intra-frame area of the current frame is the refreshed area of the current frame, i.e. the inter-frame area of the current frame.

For example, the area distribution of the refresh frame in each frame in fig. 5 also satisfies the above formula.

In addition, based on step 401, for the image blocks in the inter-frame region of the current frame, the code stream further carries MV information of these image blocks, so as to improve the decoding speed. In addition, different inter-frame coding examples are provided in step 401, and for different inter-frame coding examples, decoding modes of the decoding end are also different, which are described below correspondingly.

For the first interframe coding example in step 401, since the coding side limits MV information during coding, it is required that a reference block in a reference frame of a current frame pointed by MV information cannot fall in an un-refreshed area of the reference frame. Therefore, in this scenario, based on the area position of the forced intra area and the area position of the inter area, the implementation manner of decoding the code stream of the current frame is as follows: and if the current image block is determined to be the image block in the inter-frame area of the current frame based on the area position of the inter-frame area of the current frame, decoding the current image block directly based on the MV information of the current image block.

For the second interframe coding example in step 401, since the coding end does not limit the MV information during coding, the reference block in the reference frame of the current frame pointed by the MV information does not necessarily fall in the refreshed region of the reference frame at this time. Therefore, in order to avoid decoding failure, in this scenario, based on the area position of the forced intra-frame area and the area position of the inter-frame area, the implementation manner of decoding the code stream of the current frame is specifically: if the current image block is determined to be the image block in the inter-frame area based on the area position of the inter-frame area and the code stream carries the motion vector MV information of the current image block, adjusting the MV information to the MV information of the pointed area as the refreshed area in the reference frame; and decoding the current image block based on the adjusted MV information. That is, in such a scenario, the decoding end forces the MV information of the current block to be adjusted (clip) to the MV information pointing to the refreshed area of the reference frame, so that the adjusted MV information can point to the refreshed area of the reference frame, and thus the reference block corresponding to the current image block found in the reference frame based on the MV information can fall into the refreshed area of the reference frame.

In addition, as shown in step 401, the code stream of the current frame may further carry loop filter flag information, where the loop filter flag information indicates to turn off or allow loop filtering on a virtual boundary between the refreshed area and the non-refreshed area. The loop filtering related to the embodiment of the application comprises three loop filtering technologies: DBK (deblocking) filtering, SAO (sample adaptive offset), ALF (adaptive loop filter). Each loop filtering technique can be independently turned off or on, but on, pixels in the non-refreshed area of the current frame cannot normally be used as reference pixels.

Table 4 is a schematic table of an actual filtering operation at a decoding end under different loop filter flag information according to an embodiment of the present application.

TABLE 4

As shown in table 4, in the case that the loop filter flag information indicates to turn off the loop filter on the virtual boundary between the refreshed area and the non-refreshed area, the decoding end does not perform the operation of the loop filter on the virtual boundary between the refreshed area and the non-refreshed area.

As shown in table 4, in the case that the loop filter flag information indicates that loop filtering on the virtual boundary between the refreshed area and the non-refreshed area is allowed, the decoding end may perform loop filtering or not perform loop filtering, specifically, there are three processing manners in the following table 4.

Loop filtering example one:

in a case where the loop filter flag information indicates that loop filtering on a virtual boundary between the refreshed area and the non-refreshed area is allowed, an operation of loop filtering the area to be filtered is not performed for the area to be filtered determined based on the virtual boundary between the refreshed area and the non-refreshed area.

The region to be filtered is a region with a virtual boundary as a central line and a specified width. The specified width may be determined by a filter parameter of the loop filter, which is not limited in this embodiment of the present application. For example, the region to be filtered is an image region that extends 4 pixels left and right respectively with the virtual boundary as a center line.

Loop filter example two:

in the case where the loop filter flag information indicates that loop filtering on a virtual boundary between the refreshed area and the non-refreshed area is allowed, for a region to be filtered that is determined based on the virtual boundary between the refreshed area and the non-refreshed area, the decoding side performs an operation of loop filtering the region to be filtered based on pixels in the refreshed area.

In the second loop filtering example, the loop filtering is performed only according to the pixels in the refreshed region around the region to be filtered as the reference pixels, so that pollution caused by the loop filtering to the region which originally belongs to the refreshed region in the region to be filtered is avoided.

The reference pixels required for loop filtering may be pixels in all of the refreshed regions around the region to be filtered, or pixels in a part of the refreshed regions, which is specifically determined by parameters required for loop filtering, and this is not limited in this embodiment of the present application.

Loop filter example three:

in the case where the loop filter flag information indicates that loop filtering on a virtual boundary between the refreshed area and the non-refreshed area is allowed, for a region to be filtered determined based on the virtual boundary between the refreshed area and the non-refreshed area, the decoding side performs an operation of loop filtering the region to be filtered based on pixels in the refreshed area and pixels in the non-refreshed area. In this scenario, because the loop filtering operation pollutes the region originally belonging to the refreshed region in the region to be filtered, the region to be filtered located in the refreshed region cannot be used as the region pointed by the MV information of the image block in the next frame of the current frame in the decoding order, thereby avoiding the decoding failure of the next frame.

In the third loop filtering example, the loop filtering is performed according to the pixels in the refreshed area and the non-refreshed area around the area to be filtered simultaneously as the reference pixels, so that the area originally belonging to the refreshed area in the area to be filtered is inevitably polluted due to the loop filtering operation. The term "causing contamination" specifically means that an originally correct refreshed area is filtered based on reference pixels in the wrong non-refreshed area, thereby causing pixel errors in the originally correct refreshed area. Therefore, in this scenario, for the second inter-frame coding example in step 401, for the next frame in the decoding order, when the decoding end adjusts the MV information of an image block in the next frame, the adjusted MV information needs to point to the non-polluted refreshed area of the current frame, so as to avoid the decoding failure of the next frame.

It can be seen that, for the second inter-frame coding example in step 401, since the coding end does not limit the MV information of the image block in the inter region during the coding, the reference block in the reference frame of the image block pointed by the MV information of the image block in the inter region does not necessarily fall into the refreshed region of the reference frame. Therefore, when decoding an image block in an inter region, the decoding side has the following two processing methods.

The first method is as follows: the MV information of the image block in the inter-frame region taken by the decoding end does not limit the refreshed region of the reference frame pointing to the image block, and if the decoding end does not perform loop filtering or performs loop filtering based on the second loop filtering example when decoding the reference frame, the decoding end can directly force the MV information to point to any region in the refreshed region of the reference frame of the image block. Referring to the left reference frame of fig. 7, the MV information of the image block in the inter-frame region of the current frame may be pointed to any region in the refreshed region in the reference frame.

The second method comprises the following steps: the MV information of the image block in the inter-frame region taken by the decoding end does not limit the refreshed region of the reference frame pointing to the current frame, and if the decoding end performs loop filtering based on the loop filtering example three when decoding the reference frame, the decoding end forces the MV information of the current image block to point to the uncontaminated refreshed region of the reference frame of the current image block when decoding the current image block. Referring to the right reference frame of fig. 7, the MV information of the image block in the inter-frame region of the current frame may be pointed to the uncontaminated region in the refreshed region in the reference frame of the current image block. Wherein the range of the polluted region in the refreshed region of the reference frame does not exceed the maximum line buffer (line buffer) of the filter.

In the above, when the reference frame of the current block of the current frame is an intra-frame refresh frame and the current block is a block in an inter-frame region of the current frame, the MV information of the current image block is directly forced to point to an uncontaminated region in the refreshed region in the reference frame.

Optionally, in a case that a reference frame of a current block of a current frame is an intra-frame refresh frame and the current block is a refreshed region of the current frame, whether the current block is in an inter-frame region of the current frame or not, MV information of the current image block may also be directly forced to point to a non-contaminated region in the refreshed region in the reference frame.

Optionally, when the reference frame of the current block of the current frame is an intra-frame refresh frame and the current frame where the current block is located is not an intra-frame refresh frame, the MV information of the current image block may also be directly forced to point to an uncontaminated area in a refreshed area in the reference frame.

That is, for the current frame, when the current block of the current frame satisfies any one of the three cases, the MV information of the current image block may be directly forced to point to an uncontaminated area in the refreshed area in the reference frame. The three situations can be summarized as follows: if the reference frame of the current block is an intra-frame refresh frame and the current block does not satisfy that the current block is an adaptive intra-frame refresh frame, the current block is in an un-refreshed region, and the current block and the reference frame are in the same adaptive intra-frame refresh period, the MV information of the current block is pointed to an un-contaminated region in the refreshed region in the reference frame of the current block. Specifically, this can be achieved by the following formula:

wherein MvE is a motion vector of the current image block, MvE- > X 'is a component of the motion vector of the adjusted current image block in the X direction, MvE- > y' is a component of the motion vector of the adjusted current image block in the y direction, MvE- > X is a component of the motion vector of the current image block carried in the codestream in the X direction, MvE- > y is a component of the motion vector of the current image block carried in the codestream in the y direction, RefAirBoundX and RefAirBoundY are virtual boundary coordinates AirBoundX and AirBoundY of the reference frame, respectively, MaxFilterLineBuf is a maximum line buffer of a filter (including loop filter and ALF (adaptive loop filter)) used for loop filtering the virtual boundary of the reference frame, and PixelResolution is a pixel precision of the motion vector MvE. W is the image width of the current image block, H is the image height of the current image block, and (xE, yE) is the position of the top left corner sample of the current image block in the luminance sample matrix of the current frame. Wherein the current image block is also referred to as the current coding unit.

It should be noted that, if the reference frame of the current block of the current frame is an intra-frame refresh frame and the current block does not satisfy "the current frame is an adaptive intra-frame refresh frame, and the current block falls in an un-refreshed region, and the current frame and the reference frame are in the same adaptive intra-frame refresh period", it may also be directly forced to redirect MV information of the current image block to a refreshed region in the reference frame, and whether to direct to the refreshed region in the reference frame depends on the aforementioned manner of performing loop filtering on the virtual boundary of the reference frame.

In combination with the specific operation of the decoding end with respect to the first example of inter-frame coding in step 401 and the processing manners of the two decoding ends when decoding the image block in the inter-frame region, in the embodiment of the present application, when the decoding end performs inter-frame decoding, the following three decoding methods may be specifically used.

The method comprises the following steps: for a current image block to be decoded, the decoding end defaults that the MV information of the current image block taken from the code stream is a refreshed area pointing to a reference frame of the current image block, and the decoding end does not need any adjustment and can directly decode the current image block based on the MV information of the current image block. The method needs to limit the MV information of the current image block at the encoding end, and requires that the reference block of the current image block cannot fall in the non-refreshed area of the reference frame of the current image block.

The second method comprises the following steps: for the current image block to be decoded, the MV information of the current image block taken by the decoding end is not limited to be pointed to the refreshed area of the reference frame of the current image block, and at this time, when the decoding end obtains the reference block of the current image block, the decoding end needs to force the MV information of the current image block to be adjusted to point to the refreshed area of the reference frame of the current image block, as shown in the left reference frame of fig. 7.

The third method comprises the following steps: for the current image block to be decoded, the MV information of the current image block taken by the decoding end does not limit its pointing to the refreshed area of the reference frame of the current image block, and at this time, when the decoding end obtains the reference block of the current image block, it is forced to adjust the MV information of the current image block to point to the uncontaminated refreshed area of the reference frame of the current image block, see the reference frame on the right side of fig. 7.

In addition, in the random access scene, since the intra refresh frames (i.e. non-refresh Point frames) except the last intra refresh frame in the refresh period may not be displayed, and pixels of the non-refreshed areas of the intra refresh frames except the last intra refresh frame may not be referred to by the subsequent image frame, in order to further increase the decoding rate, some decoding processes for skipping the non-refreshed areas in the other intra refresh frames may be considered. Specifically, the explanation can be separately explained by several decoding-side skip examples described below.

Decoding-side skip decoding example one:

in a random access decoding scenario, if a current frame is in a refresh period where an access point is located and the current frame contains an un-refreshed area, indicating that the current frame is not a last frame in the refresh period, at this time, based on an area position of the refreshed area and an area position of the un-refreshed area, an implementation manner of decoding a code stream of the current frame may be: analyzing a code stream of a current LCU (large coding unit) to obtain position information of the current LCU; if the current LCU is determined to be located in the non-refreshed area of the current frame based on the position information of the current LCU, the operation of reconstructing the current LCU is not executed, and the operation of analyzing the code stream of the next LCU of the current LCU is executed; if the current LCU is located in the refreshed area, an operation of reconstructing the current LCU is performed.

In the first example of skipping decoding at the decoding end, the syntax element of each LCU needs to be parsed, and it is only clear whether the current LCU is located in the non-refreshed area of the current frame after parsing the syntax element of each LCU, and if the current LCU is located in the non-refreshed area of the current frame, the reconstruction operation of the LCU is not performed. If the current LCU is located in the refreshed area of the current frame, a reconstruction operation of the LCU is performed. In this way, only the reconstruction operation of the non-refreshed area is skipped, thereby reducing the decoding flow to improve the decoding rate.

That is, in the first skip-decoding example, when the boundary of the area to be decoded is aligned to the boundary of the LCU, which indicates that an LCU is to be decoded, the reconstruction of the LCU on the non-refreshed area in the current frame is skipped, but the parsing operation of the LCU is not skipped.

Decoding end skip decoding example two:

in a random access decoding scene, if a current frame is in a refresh cycle where an access point is located and the current frame contains an un-refreshed area, it indicates that the current frame is not a last frame in the refresh cycle, and at this time, based on an area position of the refreshed area and an area position of the un-refreshed area, a code stream of the current frame is decoded in a manner that: if the code stream start marking information of the current image slice (patch) indicates that the current image slice is located in the non-refreshed area of the current frame, the operation of analyzing the code stream of the current image slice is not executed, and the initial position of the code stream of the next image slice is obtained in the code stream of the current frame based on the code stream start marking information so as to decode the next image slice; and if the code stream start mark information of the current image slice indicates that the current image slice is located in the refreshed area, analyzing the code stream of the current image slice to decode the current image slice.

In the second example of skipping decoding at the decoding end, since the code stream start flag information corresponding to each image slice can indicate whether the current image slice is located in the non-refreshed area of the current frame, the code stream start flag information corresponding to each image slice can determine whether the current image slice is located in the non-refreshed area of the current frame, and there is no need to parse syntax elements of each LCU included in each image slice. That is, if the code stream start flag information of the current image slice (patch) indicates that the current image slice is located in the non-refreshed area of the current frame, the parsing operations of all LCUs included in the image slice are directly skipped, and the start position of the code stream of the next image slice is directly obtained in the code stream of the current frame based on the code stream start flag information, so as to decode the next code stream. And if the code stream start mark information of the current image slice indicates that the current image slice is located in the refreshed area, continuing to perform the analysis operation of all LCUs included in the image slice, so as to determine whether to reconstruct the LCUs based on the positions of the LCUs.

That is, in the second skip-decoding example, when the boundary of the region to be decoded is aligned to the boundary of an image slice (patch), which indicates that an image slice (patch) is to be decoded, the parsing and decoding operations of the image slice (patch) on the non-refreshed region in the current frame are skipped. This is done because each image slice (patch) has a code stream start mark information patch _ start _ code, and the code stream start mark information patch _ start _ code identifies the position of the image slice (patch) in the current frame and the code stream, so that the code stream parsing operation of the image slice (patch) in the non-refreshed area in the current frame can be skipped according to the code stream start mark information patch _ start _ code, and the code stream of the next image slice (patch) can be directly parsed.

Decoding end skip decoding example three:

in a random access decoding scene, if a current frame is in a refresh cycle in which an access point is located and the current frame contains an un-refreshed area, indicating that the current frame is not a last frame in the refresh cycle, at this time, based on an area position of the refreshed area and an area position of the un-refreshed area, a code stream of the current frame is decoded in a manner that: and decoding all code streams in the current frame.

That is, in the third decoding-end skip decoding example, the decoding end does not skip the parsing and reconstructing operations of the non-refreshed area in the current frame.

To sum up, the encoding and decoding method provided by the embodiment of the present application at least has the following technical effects:

(1) the virtual boundary position mark information is newly added in the code stream of the current frame, and can indicate the position of the virtual boundary between the refreshed area and the non-refreshed area in the current frame in the horizontal direction of the current frame and the position of the virtual boundary between the refreshed area and the non-refreshed area in the current frame in the vertical direction of the current frame, so that when the coding end divides the forced intra-frame area of the current frame, the virtual boundary can be arbitrarily set based on actual requirements, and then the virtual boundary can be indicated through the virtual boundary position mark information, so that the flexibility of dividing the forced intra-frame area is improved. For example, the encoding end may divide the forced intra-frame region of the current frame based on the boundary of the object included in the image content of the current frame, so that the virtual boundary between the refreshed region and the non-refreshed region may coincide with the boundary of the object in the image content of the current frame as much as possible. That is, the embodiments of the present application provide a flexible intra-frame area partition method, which can be flexibly configured by an encoding end.

(2) The decoding side may allow various ways of loop filtering across a virtual boundary between a refreshed region and a non-refreshed region, thereby improving the subjective quality of the image on the virtual boundary.

The following describes the complete decoding process at the decoding end by taking the syntax elements shown in tables 1 to 3 as examples. Specifically, the decoding flow at the decoding end includes the following steps.

Step 1: when decoding the sequence header, the decoding end makes the following judgments: if the AirEnableFlag has a value of 1, then both the values of PreAirBoundX and PreAirBoundY are initialized to 0.

Step 2: when decoding the image header of any current frame, making the following judgment: setting both the values of PreAirBoundX and PreAirBoundY to 0 if the value of PictureAirPicFlag is equal to 0; otherwise, the region positions of the respective regions are derived from PreAirBoundX, PreAirBoundY, AirBoundX, and AirBoundY based on:

1) the refreshed area is a rectangular area with (0, 0) as the upper left corner, the width of AirBoundX and the height of AirBoundY; the non-refreshed area is the remaining image area;

2) the refreshed area may be divided into an inter-frame area, which is a rectangular area with (0, 0) as the upper left corner, a width of PreAirBoundX, and a height of PreAirBoundY, and a forced intra-frame area, which is the remaining refreshed area.

And 3, step 3: when decoding the current picture, making the following decisions:

(1) if the current frame is an intra-frame refresh frame, different processing is performed on different areas, specifically as follows:

for the forced intra area, if the value of PictureAirPicFlag is equal to 1 and the current largest coding unit falls within the forced intra area, the decoding mode of each coding unit is defaulted to be the forced intra decoding mode without decoding the mode information of each coding unit therein. A sample is "unavailable" if the block in which the sample is located is "not present" or the sample has not been decoded or the sample falls within an un-refreshed region of an intra-refresh frame; otherwise this sample is "available".

For the Inter region, before deriving the prediction samples, if the reference frame of the current block is an intra-frame refresh frame and the reference block of the current block does not fall in a refreshed region of the reference frame, the MV information is adjusted so that the adjusted MV information points to the refreshed region of the reference frame.

For the non-refreshed area, the decoding end does not perform special processing.

(2) If the current frame is not the intra-frame refreshing frame, the image blocks in the intra-frame area of the current frame are not specially processed, and the following judgment is made on the image blocks in the inter-frame area of the current frame:

before deriving the prediction samples, if the reference frame of the current block is an intra-frame refresh frame and the reference block of the current block is not located in a refreshed region of the reference frame, the MV information of the current block is adjusted so that the adjusted MV information points to the refreshed region of the reference frame.

And 4, step 4: no special processing is done to the loop filtering process.

And 5: after the current image is decoded, the following judgment is made: if the value of PictureAirPicFlag is equal to 1, the values of PreAirBoundX and PreAirBoundY are updated to the values of AirBoundX and AirBoundY, respectively.

And 6: and (3) turning to the step 2, and continuing to decode the next frame of image until the current sequence is decoded completely.

It should be noted that the above steps 1 to 6 are only optional exemplary steps in the embodiment shown in fig. 6, and do not constitute a limitation to the specific step of decoding the current frame at the decoding end. When the embodiment of the present application is applied, the decoding end may combine according to any optional example in the embodiment of fig. 6 to completely decode the current frame. That is, all the above optional technical solutions may be combined to form an optional embodiment of the present application at will, and details of the embodiment of the present application are not described herein any more.

Fig. 8 is a schematic structural diagram of a decoding apparatus according to an embodiment of the present application. As shown in fig. 8, the decoding apparatus 800 includes the following modules.

A receiving module 801, configured to receive a code stream of a current frame, where the code stream of the current frame carries frame type flag information, and the frame type flag information indicates whether the current frame is an intra-frame refresh frame;

a processing module 802, configured to obtain virtual boundary position flag information carried in a code stream when the frame type flag information indicates that the current frame is an intra-frame refresh frame, where the virtual boundary position flag information indicates a position of a virtual boundary, and the virtual boundary is at least used to distinguish a refreshed region from an un-refreshed region in the current frame;

the processing module is further used for determining the area position of the refreshed area and the area position of the non-refreshed area based on the virtual boundary position mark information;

and the processing module is also used for decoding the code stream of the current frame based on the area position of the refreshed area and the area position of the non-refreshed area.

In one possible implementation, the virtual boundary position marker information includes first virtual boundary position marker information and second virtual boundary position marker information, the first virtual boundary position marker information indicating a position of the virtual boundary in a horizontal direction of the current frame, the second virtual boundary position marker information indicating a position of the virtual boundary in a vertical direction of the current frame;

wherein, the position of the virtual boundary in the current frame in the vertical direction of the current frame is different from the position of the virtual boundary between the refreshed area and the non-refreshed area in the adjacent frame in the vertical direction of the adjacent frame, and the adjacent frame is a previous frame or a next frame of the current frame in the decoding sequence.

In a possible implementation manner, the first virtual boundary position mark information is a pixel position of the virtual boundary in a horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of the virtual boundary in a vertical direction of the current frame.

In a possible implementation manner, the first virtual boundary position mark information is an image abscissa of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is an image ordinate of the virtual boundary in the vertical direction of the current frame;

the refreshed area is a rectangular area with the coordinates (0, 0) at the upper left corner of the image as the origin, the width as the abscissa of the image and the height as the ordinate of the image, and the non-refreshed area is an area other than the refreshed area in the current frame.

In one possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in the horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in the vertical direction of the current frame and a height of the current frame image.

In one possible implementation, the width precision of the refreshed region is the width of one coding unit CU, and the height precision of the refreshed region is the height of the CU; alternatively, the first and second electrodes may be,

the width accuracy of the refreshed area is the width of one image slice, and the height accuracy of the refreshed area is the height of the image slice, and the image slice comprises one or more LCUs.

In one possible implementation, the refreshed region includes a forced intra region and an inter region;

the processing module is used for:

determining the region position of the forced intra region and the region position of the inter region based on the region position of the refreshed region and the refreshed region of the previous frame of the current frame in decoding order;

In one possible implementation, the inter-frame region is a refreshed region of a previous frame;

the intra-frame region is forced to be a region other than the inter-frame region in the refreshed region of the current frame.

In one possible implementation, the processing module is configured to:

In one possible implementation, the processing module is further configured to:

if the reference frame of the current image block to be decoded in the current frame is an intra-frame refreshing frame and the current image block does not meet the condition that the current frame is an intra-frame refreshing frame, the current image block falls in an un-refreshed area and the current frame and the reference frame are in the same refreshing cycle, adjusting the MV information of the current image block to be the MV information of the refreshed area in the reference frame;

and decoding the current image block based on the adjusted MV information.

In one possible implementation, the adjusted MV information satisfies the following condition:

wherein MvE is a motion vector of the current image block, MvE- > X 'is a component of the motion vector of the adjusted current image block in the X direction, MvE- > y' is a component of the motion vector of the adjusted current image block in the y direction, MvE- > X is a component of the motion vector of the current image block carried in the code stream in the X direction, MvE- > y is a component of the motion vector of the current image block carried in the code stream in the y direction, RefAirBoundX and RefAirBoundY are virtual boundary coordinates AirBoundX and AirBoundY of the reference frame, MaxFilterLineBuf is a maximum line buffer of a filter used for loop filtering a virtual boundary of the reference frame, PixelResolution is a pixel precision of the motion vector, W is an image width of the current image block, H is an image height of the current image block, and xE, yE is a position of an upper left corner sample in a luminance sample matrix of the current frame.

In a possible implementation manner, the code stream carries loop filtering flag information, and the loop filtering flag information indicates that loop filtering on a virtual boundary is closed or allowed.

In one possible implementation, in a case that the loop filtering flag information indicates that loop filtering on the virtual boundary is allowed, the processing module is further configured to:

and for the region to be filtered determined based on the virtual boundary, not executing the operation of loop filtering on the region to be filtered.

and for the region to be filtered determined based on the virtual boundary, performing loop filtering operation on the region to be filtered based on the pixels in the refreshed region.

and for the region to be filtered determined based on the virtual boundary, performing loop filtering operation on the region to be filtered based on the pixels in the refreshed region and the pixels in the non-refreshed region, wherein the region to be filtered positioned in the refreshed region cannot be used as the region pointed by the MV information of the image block in the next frame of the current frame in the decoding order.

In a possible implementation manner, in a scenario of random access decoding, if a current frame is in a refresh period where an access point is located and the current frame contains an un-refreshed area, the processing module is configured to:

analyzing the code stream of the current LCU to obtain the position information of the current LCU;

if the current LCU is located in the refreshed area, an operation of reconstructing the current LCU is performed.

and decoding all code streams in the current frame.

In summary, in the embodiment of the present application, virtual boundary position flag information is newly added in a code stream of a current frame, and the virtual boundary position flag information can indicate a position of a virtual boundary between a refreshed region and an un-refreshed region in the current frame in a horizontal direction of the current frame and a position of a virtual boundary between a refreshed region and an un-refreshed region in the current frame in a vertical direction of the current frame, so that when an encoding end divides a forced intra-frame region of the current frame, the virtual boundary can be arbitrarily set based on an actual requirement, and then the virtual boundary is indicated by the virtual boundary position flag information, so that flexibility of dividing the forced intra-frame region is improved. For example, the encoding end may divide the forced intra-frame region of the current frame based on the boundary of the object included in the image content of the current frame, so that the virtual boundary between the refreshed region and the non-refreshed region may coincide with the boundary of the object in the image content of the current frame as much as possible. That is, the embodiments of the present application provide a flexible intra region partition method, which can be flexibly configured by an encoding end.

Secondly, the decoding end can allow loop filtering in various ways across a virtual boundary between a refreshed region and an un-refreshed region, thereby improving the subjective quality of the image on the virtual boundary.

It should be noted that: when the decoding end provided in the foregoing embodiment implements the decoding method based on the adaptive intra-frame refresh mechanism, only the division of the functional modules is illustrated, and in practical applications, the above functions may be distributed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the above described functions. In addition, the decoding end provided by the above embodiment and the decoding method embodiment based on the adaptive intra refresh mechanism belong to the same concept, and the specific implementation process thereof is described in detail in the method embodiment and is not described herein again.

Fig. 9 is a schematic structural diagram of an encoding apparatus according to an embodiment of the present application, and as shown in fig. 9, the encoding apparatus 900 includes the following modules.

A sending module 901, configured to send a code stream of a current frame;

the code stream of the current frame carries frame type mark information, the frame type mark information indicates whether the current frame is an intra-frame refresh frame, the code stream also carries virtual boundary position mark information under the condition that the frame type mark information indicates that the current frame is the intra-frame refresh frame, the virtual boundary position mark information indicates the position of a virtual boundary, and the virtual boundary is at least used for distinguishing a refreshed area and an un-refreshed area in the current frame.

the position of the virtual boundary in the current frame in the vertical direction of the current frame is different from the position of the virtual boundary between the refreshed area and the non-refreshed area in the adjacent frame in the vertical direction of the adjacent frame, and the adjacent frame is a previous frame or a next frame of the current frame in the coding sequence.

In one possible implementation manner, the first virtual boundary position mark information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position mark information is a pixel position of the virtual boundary in the vertical direction of the current frame.

the refreshed area is a rectangular area with coordinates (0, 0) at the upper left corner of the image as an origin, the width as the horizontal coordinate of the image and the height as the vertical coordinate of the image, and the non-refreshed area is an area other than the refreshed area in the current frame.

In a possible implementation manner, the first virtual boundary position flag information is a ratio between a width of the refreshed region in a horizontal direction of the current frame and a width of the current frame image, and the second position flag information is a ratio between a height of the refreshed region in a vertical direction of the current frame and a height of the current frame image.

In a possible implementation manner, the refreshed area includes a forced intra-frame area and an inter-frame area, and the motion vector MV information of the image block in the inter-frame area carried in the code stream points to the refreshed area in the reference frame of the image block.

In a possible implementation manner, the refreshed area includes a forced intra-frame area and an inter-frame area, and the motion vector MV information of the image block in the inter-frame area carried in the code stream points to the refreshed area in the reference frame of the image block and/or the non-refreshed area in the reference frame of the image block.

In a possible implementation manner, the code stream carries loop filtering flag information, and the loop filtering flag information indicates that loop filtering on a virtual boundary is turned off or allowed.

In summary, the virtual boundary position mark information is newly added in the code stream of the current frame, and the virtual boundary position mark information can indicate the position of the virtual boundary, and the virtual boundary is at least used for distinguishing the refreshed region from the non-refreshed region in the current frame, so that when the coding end divides the forced intra-frame region of the current frame, the virtual boundary can be arbitrarily set based on actual requirements, and then the position of the virtual boundary is indicated by the virtual boundary position mark information, so that the flexibility of dividing the forced intra-frame region is improved, that is, the embodiment of the present application provides an adaptive intra-frame refreshing mechanism for adaptively dividing the forced intra-frame region. For example, the encoding end may divide the forced intra-frame region of the current frame based on the boundary of the object included in the image content of the current frame, so that the virtual boundary between the refreshed region and the non-refreshed region may coincide with the boundary of the object in the image content of the current frame as much as possible.

It should be noted that: when the encoding end provided in the foregoing embodiment implements the encoding method based on the adaptive intra-frame refresh mechanism, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the encoding end provided by the above embodiment and the encoding method embodiment based on the adaptive intra refresh mechanism belong to the same concept, and the specific implementation process thereof is described in detail in the method embodiment and is not described herein again.

Fig. 10 is a block diagram of a terminal 1000 according to an embodiment of the present application. The decoding end device, the decoding end, the decoding apparatus, the encoding end device, the encoding end, and the encoding apparatus related to the foregoing embodiments can all be implemented by the terminal. Specifically, the terminal 1000 can be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Terminal 1000 can also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or the like by other names.

In general, terminal 1000 can include: a processor 1001 and a memory 1002.

Processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 1001 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 1001 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in a wake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor 1001 may further include an AI (Artificial Intelligence) processor for processing a computing operation related to machine learning.

Memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 1002 is configured to store at least one instruction for execution by the processor 1001 to implement the adaptive intra refresh mechanism based decoding and encoding methods provided by the method embodiments of the present application.

In some embodiments, terminal 1000 can also optionally include: a peripheral interface 1003 and at least one peripheral. The processor 1001, memory 1002 and peripheral interface 1003 may be connected by a bus or signal line. Various peripheral devices may be connected to peripheral interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 1004, display screen 1005, camera assembly 1006, audio circuitry 1007, positioning assembly 1008, and power supply 1009.

The peripheral interface 1003 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 1001 and the memory 1002. In some embodiments, processor 1001, memory 1002, and peripheral interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, the memory 1002, and the peripheral interface 1003 may be implemented on separate chips or circuit boards, which are not limited by this embodiment.

The Radio Frequency circuit 1004 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 1004 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 1004 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1004 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 1004 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 1004 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 1005 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 1005 is a touch display screen, the display screen 1005 also has the ability to capture touch signals on or over the surface of the display screen 1005. The touch signal may be input to the processor 1001 as a control signal for processing. At this point, the display screen 1005 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, display screen 1005 can be one, providing a front panel of terminal 1000; in other embodiments, display 1005 can be at least two, respectively disposed on different surfaces of terminal 1000 or in a folded design; in other embodiments, display 1005 can be a flexible display disposed on a curved surface or a folded surface of terminal 1000. Even more, the display screen 1005 may be arranged in a non-rectangular irregular figure, i.e., a shaped screen. The Display screen 1005 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and the like.

The camera assembly 1006 is used to capture images or video. Optionally, the camera assembly 1006 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 1006 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuit 1007 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals into the processor 1001 for processing or inputting the electric signals into the radio frequency circuit 1004 for realizing voice communication. For stereo sound collection or noise reduction purposes, multiple microphones can be provided, each at a different location of terminal 1000. The microphone may also be an array microphone or an omni-directional acquisition microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The loudspeaker can be a traditional film loudspeaker and can also be a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuit 1007 may also include a headphone jack.

A Location component 1008 is employed to locate a current geographic Location of terminal 1000 for purposes of navigation or LBS (Location Based Service). The Positioning component 1008 may be a Positioning component based on the Global Positioning System (GPS) in the united states, the beidou System in china, the graves System in russia, or the galileo System in the european union.

Power supply 1009 is used to supply power to various components in terminal 1000. The power source 1009 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When the power source 1009 includes a rechargeable battery, the rechargeable battery may support wired charging or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 1000 can also include one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: acceleration sensor 1011, gyro sensor 1012, pressure sensor 1013, fingerprint sensor 1014, optical sensor 1015, and proximity sensor 1016.

Acceleration sensor 1011 can detect acceleration magnitudes on three coordinate axes of a coordinate system established with terminal 1000. For example, the acceleration sensor 1011 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 1001 may control the display screen 1005 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 1011. The acceleration sensor 1011 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 1012 may detect a body direction and a rotation angle of the terminal 1000, and the gyro sensor 1012 and the acceleration sensor 1011 may cooperate to acquire a 3D motion of the user on the terminal 1000. From the data collected by the gyro sensor 1012, the processor 1001 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensor 1013 can be disposed on a side frame of terminal 1000 and/or underneath display screen 1005. When pressure sensor 1013 is disposed on a side frame of terminal 1000, a user's grip signal on terminal 1000 can be detected, and processor 1001 performs left-right hand recognition or shortcut operation according to the grip signal collected by pressure sensor 1013. When the pressure sensor 1013 is disposed at a lower layer of the display screen 1005, the processor 1001 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 1005. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 1014 is used to collect a fingerprint of the user, and the processor 1001 identifies the user according to the fingerprint collected by the fingerprint sensor 1014, or the fingerprint sensor 1014 identifies the user according to the collected fingerprint. Upon identifying that the user's identity is a trusted identity, the processor 1001 authorizes the user to perform relevant sensitive operations including unlocking a screen, viewing encrypted information, downloading software, paying, and changing settings, etc. Fingerprint sensor 1014 can be disposed on the front, back, or side of terminal 1000. When a physical key or vendor Logo is provided on terminal 1000, fingerprint sensor 1014 can be integrated with the physical key or vendor Logo.

Optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 may control the display brightness of the display screen 1005 according to the ambient light intensity collected by the optical sensor 1015. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is turned down. In another embodiment, the processor 1001 may also dynamically adjust the shooting parameters of the camera assembly 1006 according to the intensity of the ambient light collected by the optical sensor 1015.

Proximity sensor 1016, also known as a distance sensor, is typically disposed on a front panel of terminal 1000. Proximity sensor 1016 is used to gather the distance between the user and the front face of terminal 1000. In one embodiment, when proximity sensor 1016 detects that the distance between the user and the front surface of terminal 1000 is gradually reduced, processor 1001 controls display screen 1005 to switch from a bright screen state to a dark screen state; when proximity sensor 1016 detects that the distance between the user and the front of terminal 1000 is gradually increased, display screen 1005 is controlled by processor 1001 to switch from a breath-screen state to a bright-screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 10 is not limiting of terminal 1000 and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components may be employed.

The embodiment of the present application further provides a non-transitory computer-readable storage medium, and when instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the decoding and encoding method based on the adaptive intra refresh mechanism provided in the above embodiment.

The embodiment of the present application further provides a computer program product containing instructions, which when run on a terminal, causes the terminal to execute the decoding and encoding method based on the adaptive intra refresh mechanism provided in the foregoing embodiment.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A decoding method based on an adaptive intra refresh mechanism, the method comprising:

determining an area position of the refreshed area and an area position of the non-refreshed area based on the virtual boundary position marker information;

2. The method of claim 1, wherein the virtual boundary position marker information includes first virtual boundary position marker information indicating a position of the virtual boundary in the current frame horizontal direction and second virtual boundary position marker information indicating a position of the virtual boundary in the current frame vertical direction;

3. The method of claim 2, wherein the first virtual boundary position flag information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position flag information is a pixel position of the virtual boundary in the vertical direction of the current frame.

4. The method according to claim 3, wherein the first virtual boundary position marker information is an image abscissa of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position marker information is an image ordinate of the virtual boundary in the vertical direction of the current frame;

the refreshed region is a rectangular region with coordinates (0, 0) at the upper left corner of the image as an origin, the width as the abscissa and the height as the ordinate of the image, and the non-refreshed region is the other region except the refreshed region in the current frame.

5. The method of claim 2, wherein the first virtual boundary position flag information is a ratio between a width of the refreshed region in the current frame horizontal direction and the current frame image width, and the second position flag information is a ratio between a height of the refreshed region in the current frame vertical direction and the current frame image height.

6. The method of claim 1,

the width precision of the refreshed area is the width of one Coding Unit (CU), and the height precision of the refreshed area is the height of the CU; alternatively, the first and second electrodes may be,

the width accuracy of the refreshed area is the width of an image slice, the height accuracy of the refreshed area is the height of the image slice, and the image slice comprises one or more LCUs.

7. The method of claim 1, wherein the refreshed regions comprise forced intra regions and inter regions;

determining a region position of the forced intra region and a region position of the inter region based on a region position of the refreshed region and a refreshed region of a previous frame of the current frame in a decoding order;

8. The method of claim 7,

the inter-frame region is a refreshed region of the previous frame;

9. The method of claim 7,

decoding the code stream of the current frame based on the area position of the forced intra-frame area and the area position of the inter-frame area, including:

10. The method of claim 1, wherein after receiving the codestream of the current frame, the method further comprises:

and decoding the current image block based on the adjusted MV information.

11. The method of claim 10, wherein the adjusted MV information satisfies the following condition:

12. The method of claim 1, wherein the code stream carries loop filter flag information indicating that loop filtering on the virtual boundary is turned off or allowed.

13. The method of claim 12, wherein in a case that the loop filter flag information indicates that loop filtering on the virtual boundary is allowed, the method further comprises:

and for the region to be filtered which is determined based on the virtual boundary, not performing the operation of loop filtering on the region to be filtered.

14. The method of claim 12, wherein in a case that the loop filter flag information indicates that loop filtering on the virtual boundary is allowed, the method further comprises:

15. The method of claim 12, wherein in a case that the loop filter flag information indicates that loop filtering on the virtual boundary is allowed, the method further comprises:

16. The method of claim 1, wherein in a scenario of random access decoding, if the current frame is in a refresh period in which an access point is located and the current frame contains an un-refreshed region, the decoding a code stream of the current frame based on a region position of the refreshed region and a region position of the un-refreshed region comprises:

17. The method of claim 1, wherein in a scenario of random access decoding, if the current frame is in a refresh period in which an access point is located and the current frame contains an un-refreshed region, the decoding a code stream of the current frame based on a region position of the refreshed region and a region position of the un-refreshed region comprises:

18. The method of claim 1, wherein in a scenario of random access decoding, if the current frame is in a refresh period in which an access point is located and the current frame contains an un-refreshed region, the decoding a code stream of the current frame based on a region position of the refreshed region and a region position of the un-refreshed region comprises:

and decoding all code streams in the current frame.

19. An encoding method based on an adaptive intra refresh mechanism, the method comprising:

transmitting a code stream of the current frame;

20. The method of claim 19, wherein the virtual boundary position flag information includes first virtual boundary position flag information indicating a position of the virtual boundary in the current frame horizontal direction and second virtual boundary position flag information indicating a position of the virtual boundary in the current frame vertical direction;

21. The method of claim 20, wherein the first virtual boundary position flag information is a pixel position of the virtual boundary in the horizontal direction of the current frame, and the second virtual boundary position flag information is a pixel position of the virtual boundary in the vertical direction of the current frame.

22. The method of claim 21, wherein the first virtual boundary position marker information is an image abscissa of the virtual boundary in the current frame horizontal direction, and the second virtual boundary position marker information is an image ordinate of the virtual boundary in the current frame vertical direction;

23. The method of claim 20, wherein the first virtual boundary position flag information is a ratio between a width of the refreshed region in the current frame horizontal direction and the current frame image width, and the second position flag information is a ratio between a height of the refreshed region in the current frame vertical direction and the current frame image height.

24. The method of claim 19,

25. The method of claim 19, wherein the refreshed areas comprise a forced intra area and an inter area, and motion vector MV information of an image block in the inter area carried in the code stream points to a refreshed area in a reference frame of the image block.

26. The method of claim 19, wherein the refreshed areas comprise a forced intra area and an inter area, and motion vector MV information of the image block in the inter area carried in the code stream points to a refreshed area in a reference frame of the image block and/or an un-refreshed area in a reference frame of the image block.

27. The method of claim 19, wherein the code stream carries loop filter flag information indicating that loop filtering on the virtual boundary is turned off or allowed.

28. A decoding apparatus, characterized in that the decoding apparatus comprises:

a processing module, configured to obtain, when the frame type flag information indicates that the current frame is an intra-frame refresh frame, virtual boundary position flag information carried in the code stream, where the virtual boundary position flag information indicates a position of a virtual boundary, and the virtual boundary is at least used to distinguish a refreshed area from an un-refreshed area in the current frame;

29. An encoding apparatus, characterized in that the encoding apparatus comprises:

the sending module is used for sending the code stream of the current frame;

30. A decoding-side apparatus, characterized in that the decoding-side apparatus comprises:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of the method of any of the preceding claims 1 to 18.

31. An encoding side device, characterized in that the encoding side device comprises:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of the method of any of the above claims 19 to 27.

32. A computer-readable storage medium having stored thereon instructions which, when executed by a processor, carry out the steps of the method of any of the preceding claims 1 to 18, or 19 to 27.