WO2024007157A1 - 多参考行索引列表排序方法、视频编解码方法、装置和系统 - Google Patents

多参考行索引列表排序方法、视频编解码方法、装置和系统 Download PDF

Info

Publication number
WO2024007157A1
WO2024007157A1 PCT/CN2022/103944 CN2022103944W WO2024007157A1 WO 2024007157 A1 WO2024007157 A1 WO 2024007157A1 CN 2022103944 W CN2022103944 W CN 2022103944W WO 2024007157 A1 WO2024007157 A1 WO 2024007157A1
Authority
WO
WIPO (PCT)
Prior art keywords
current block
mrl
index list
index
mode
Prior art date
Application number
PCT/CN2022/103944
Other languages
English (en)
French (fr)
Inventor
徐陆航
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to PCT/CN2022/103944 priority Critical patent/WO2024007157A1/zh
Priority to TW112124425A priority patent/TW202404366A/zh
Publication of WO2024007157A1 publication Critical patent/WO2024007157A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters

Definitions

  • Embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to a multi-reference line index list sorting method, video encoding and decoding method, device and system.
  • Digital video compression technology mainly compresses huge digital image and video data to facilitate transmission and storage.
  • Current common video encoding and decoding standards such as H.266/Versatile Video Coding (VVC), all use block-based hybrid coding frameworks.
  • Each frame in the video is divided into square largest coding units (LCU: largest coding unit) of the same size (such as 128x128, 64x64, etc.).
  • Each maximum coding unit can be divided into rectangular coding units (CU: coding unit) according to rules.
  • Coding units may also be divided into prediction units (PU: prediction unit), transformation units (TU: transform unit), etc.
  • the hybrid coding framework includes prediction, transform, quantization, entropy coding, in loop filter and other modules.
  • the prediction module includes intra prediction and inter prediction, which are used to reduce or remove the inherent redundancy of the video.
  • Intra-frame blocks are predicted using the surrounding pixels of the block as a reference, while inter-frame blocks refer to spatially adjacent block information and reference information in other frames.
  • the residual information is encoded into a code stream through block-based transformation, quantization and entropy encoding.
  • An embodiment of the present disclosure provides a multi-reference row index list sorting method, including:
  • the index of the reference row corresponding to the error is filled into the MRL index list of the current block to obtain an MRL index list sorted based on the current block.
  • An embodiment of the present disclosure also provides a video decoding method, including:
  • sort the MRL index list of the current block When it is determined according to the decoding result that sorting of the MRL index list of the current block is allowed, sort the MRL index list of the current block according to the method described in any embodiment of the present disclosure, and obtain an MRL index list sorted based on the current block;
  • the reference row selected by the current block is determined, and the current block is intra-predicted according to the reference row selected by the current block.
  • An embodiment of the present disclosure also provides a video encoding method, including:
  • sort the MRL index list of the current block When it is determined that sorting of the MRL index list of the current block is allowed, sort the MRL index list of the current block according to the method described in any embodiment of the present disclosure, and obtain an MRL index list sorted based on the current block;
  • the multi-reference row index of the current block is determined and encoded.
  • An embodiment of the present disclosure also provides a code stream, wherein the code stream includes a multi-reference line index of the current block, and the multi-reference line index is used to indicate that the index of the reference line selected in the current block is in the MRL index list.
  • position in, the MRL index list is the MRL index list sorted based on the current block, or the set MRL index list.
  • An embodiment of the present disclosure also provides a multi-reference row index list sorting device, which includes a processor and a memory storing a computer program. When the processor executes the computer program, it can implement any embodiment of the present disclosure.
  • the multi-reference row index list sorting method includes a processor and a memory storing a computer program.
  • An embodiment of the present disclosure also provides a video decoding device, which includes a processor and a memory storing a computer program, wherein when the processor executes the computer program, the video decoding device as described in any embodiment of the present disclosure can be realized. Decoding method.
  • An embodiment of the present disclosure also provides a video encoding device, including a processor and a memory storing a computer program, wherein when the processor executes the computer program, the video encoding device as described in any embodiment of the present disclosure can be implemented. Encoding method.
  • An embodiment of the present disclosure also provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements any implementation of the present disclosure.
  • Figure 1A is a schematic diagram of a coding and decoding system according to an embodiment of the present disclosure
  • Figure 1B is a frame diagram of the encoding end according to an embodiment of the present disclosure.
  • Figure 1C is a frame diagram of the decoding end according to an embodiment of the present disclosure.
  • Figure 2 is a schematic diagram of an intra prediction mode according to an embodiment of the present disclosure
  • Figure 3 is a schematic diagram of adjacent intra prediction blocks of the current block according to an embodiment of the present disclosure
  • Figure 4 is a schematic diagram of the template area and template reference area of the current block according to an embodiment of the present disclosure
  • Figure 5 is a schematic diagram of multiple reference lines around the current block according to an embodiment of the present disclosure.
  • Figure 6 is a flow chart of a multi-reference row index list sorting method according to an embodiment of the present disclosure
  • Figure 7 is a flow chart of a video decoding method according to an embodiment of the present disclosure.
  • Figure 8 is a flow chart of a video encoding method according to an embodiment of the present disclosure.
  • Figure 9 is a schematic diagram of a multi-reference row index list sorting device according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of a template and multiple reference lines around a current block according to an embodiment of the present disclosure.
  • the words “exemplary” or “such as” are used to mean an example, illustration, or explanation. Any embodiment described in this disclosure as “exemplary” or “such as” is not intended to be construed as preferred or advantageous over other embodiments.
  • "And/or” in this article is a description of the relationship between associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and they exist alone. B these three situations.
  • "Plural” means two or more than two.
  • words such as “first” and “second” are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as “first” and “second” do not limit the number and execution order, and words such as “first” and “second” do not limit the number and execution order.
  • the local illumination compensation method and video encoding and decoding method proposed by the embodiments of the present disclosure can be applied to various video encoding and decoding standards, such as: H.264/Advanced Video Coding (AVC), H.265/High Efficiency Video Coding (High Efficiency Video Coding, HEVC), H.266/Versatile Video Coding (VVC), AVS (Audio Video coding Standard, audio and video coding standard), and MPEG (Moving Picture Experts Group, Moving Picture Experts Group) , other standards formulated by AOM (Alliance for Open Media), JVET (Joint Video Experts Team) and extensions of these standards, or any other customized standards, etc.
  • FIG. 1A is a block diagram of a video encoding and decoding system that can be used in embodiments of the present disclosure. As shown in the figure, the system is divided into an encoding end device 1 and a decoding end device 2.
  • the encoding end device 1 generates a code stream.
  • the decoding end device 2 can decode the code stream.
  • the decoding end device 2 can receive the code stream from the encoding end device 1 via the link 3 .
  • Link 3 includes one or more media or devices capable of moving the code stream from the encoding end device 1 to the decoding end device 2 .
  • the link 3 includes one or more communication media that enable the encoding end device 1 to directly send the code stream to the decoding end device 2 .
  • the encoding end device 1 modulates the code stream according to the communication standard (such as a wireless communication protocol), and sends the modulated code stream to the decoding end device 2 .
  • the one or more communication media may include wireless and/or wired communication media and may form part of a packet network.
  • the code stream can also be output from the output interface 15 to a storage device, and the decoding end device 2 can read the stored data from the storage device via streaming or downloading.
  • the code end device 1 includes a data source 11, a video encoding device 13 and an output interface 15.
  • Data sources 11 include a video capture device (eg, a video camera), an archive containing previously captured data, a feed interface to receive data from a content provider, a computer graphics system to generate the data, or a combination of these sources.
  • the video encoding device 13 encodes the data from the data source 11 and outputs the data to the output interface 15.
  • the output interface 15 may include at least one of a regulator, a modem and a transmitter.
  • the decoding end device 2 includes an input interface 21 , a video decoding device 23 and a display device 25 .
  • the input interface 21 includes at least one of a receiver and a modem.
  • the input interface 21 may receive the code stream via link 3 or from a storage device.
  • the video decoding device 23 decodes the received code stream.
  • the display device 25 is used to display the decoded data.
  • the display device 25 can be integrated with other devices of the decoding end device 2 or set up separately.
  • the display device 25 is optional for the decoding end. In other examples, the decoding end may include other devices or devices that apply decoded data.
  • FIG. 1B is a block diagram of an exemplary video encoding device that can be used in embodiments of the present disclosure.
  • the video encoding device 1000 includes a prediction unit 1100, a division unit 1101, a residual generation unit 1102 (indicated by a circle with a plus sign after the division unit 1101 in the figure), a transformation processing unit 1104, a quantization unit 1106, Inverse quantization unit 1108, inverse transform processing unit 1110, reconstruction unit 1112 (indicated by a circle with a plus sign after the inverse transform processing unit 1110 in the figure), filter unit 1113, decoded image buffer 1114, and entropy encoding unit 1115.
  • the prediction unit 1100 includes an inter prediction unit 1121 and an intra prediction unit 1126, and the decoded image buffer 1114 may also be called a decoded image buffer, a decoded picture buffer, a decoded picture buffer, etc.
  • Video encoder 20 may also include more, fewer, or different functional components than this example, such that transform processing unit 1104, inverse transform processing unit 1110, etc. may be eliminated in some cases.
  • the dividing unit 1101 cooperates with the prediction unit 1100 to divide the received video data into slices, coding tree units (CTU: Coding Tree Unit) or other larger units.
  • the video data received by the dividing unit 1101 may be a video sequence including video frames such as I frames, P frames, or B frames.
  • the prediction unit 1100 can divide the CTU into coding units (CU: Coding Unit), and perform intra prediction encoding or inter prediction encoding on the CU.
  • CU Coding Unit
  • the CU can be divided into one or more prediction units (PU: prediction unit).
  • the inter prediction unit 1121 may perform inter prediction on the PU to generate prediction data for the PU, including prediction blocks of the PU, motion information of the PU, and various syntax elements.
  • the inter prediction unit 1121 may include a motion estimation (ME: motion estimation) unit and a motion compensation (MC: motion compensation) unit.
  • the motion estimation unit may be used for motion estimation to generate motion vectors, and the motion compensation unit may be used to obtain or generate prediction blocks based on the motion vectors.
  • Intra prediction unit 1126 may perform intra prediction on the PU to generate prediction data for the PU.
  • the prediction data of the PU may include the prediction block of the PU and various syntax elements.
  • Residual generation unit 1102 may generate a residual block of the CU based on the original block of the CU minus the prediction blocks of the PU into which the CU is divided.
  • the transformation processing unit 1104 may divide the CU into one or more transformation units (TU: Transform Unit), and the divisions of prediction units and transformation units may be different.
  • the residual block associated with the TU is the sub-block obtained by dividing the residual block of the CU.
  • a TU-associated coefficient block is generated by applying one or more transforms to the TU-associated residual block.
  • the quantization unit 1106 can quantize the coefficients in the coefficient block based on the selected quantization parameter, and the degree of quantization of the coefficient block can be adjusted by adjusting the quantization parameter (QP: Quantizer Parameter).
  • QP Quantizer Parameter
  • Inverse quantization unit 1108 and inverse transform unit 1110 may apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a TU-associated reconstructed residual block.
  • the reconstruction unit 1112 may add the reconstruction residual block and the prediction block generated by the prediction unit 1100 to generate a reconstructed image.
  • the filter unit 1113 performs loop filtering on the reconstructed image, and stores the filtered reconstructed image in the decoded image buffer 1114 as a reference image.
  • Intra prediction unit 1126 may extract reference images of blocks adjacent to the PU from decoded image buffer 1114 to perform intra prediction.
  • the inter prediction unit 1121 may perform inter prediction on the PU of the current frame image using the reference image of the previous frame buffered by the decoded image buffer 1114 .
  • the entropy encoding unit 1115 may perform an entropy encoding operation on received data (such as syntax elements, quantized coefficient blocks, motion information, etc.).
  • the video decoding device 101 includes an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform processing unit 156, and a reconstruction unit 158 (indicated by a circle with a plus sign after the inverse transform processing unit 155 in the figure). ), filter unit 159, and decoded image buffer 160.
  • the video decoder 30 may include more, fewer, or different functional components, such as the inverse transform processing unit 155 may be eliminated in some cases.
  • the entropy decoding unit 150 may perform entropy decoding on the received code stream, and extract syntax elements, quantized coefficient blocks, motion information of the PU, etc.
  • the prediction unit 152, the inverse quantization unit 154, the inverse transform processing unit 156, the reconstruction unit 158 and the filter unit 159 may all perform corresponding operations based on syntax elements extracted from the code stream.
  • Inverse quantization unit 154 may inversely quantize the quantized TU-associated coefficient block.
  • Inverse transform processing unit 156 may apply one or more inverse transforms to the inverse quantized coefficient block to produce a reconstructed residual block of the TU.
  • Prediction unit 152 includes inter prediction unit 162 and intra prediction unit 164 .
  • intra prediction unit 164 may determine the intra prediction mode of the PU based on the syntax elements decoded from the codestream, based on the determined intra prediction mode and the PU's neighbors obtained from decoded image buffer 160 Intra prediction is performed on the reconstructed reference information to generate the prediction block of the PU.
  • inter prediction unit 162 may determine one or more reference blocks for the PU based on the motion information of the PU and corresponding syntax elements, generated based on the reference blocks obtained from decoded image buffer 160 Prediction block of PU.
  • Reconstruction unit 158 may obtain a reconstructed image based on the reconstruction residual block associated with the TU and the prediction block of the PU generated by prediction unit 152 .
  • the filter unit 159 may perform loop filtering on the reconstructed image, and the filtered reconstructed image is stored in the decoded image buffer 160 .
  • the decoded image buffer 160 can provide a reference image for subsequent motion compensation, intra-frame prediction, inter-frame prediction, etc., and can also output the filtered reconstructed image as decoded video data for presentation on the display device.
  • a frame of image is divided into blocks, and intra-frame prediction or inter-frame prediction or other algorithms are performed on the current block to generate the prediction of the current block.
  • Block use the original block of the current block to subtract the prediction block to obtain the residual block, transform and quantize the residual block to obtain the quantization coefficient, and perform entropy encoding on the quantization coefficient to generate a code stream.
  • intra-frame prediction or inter-frame prediction is performed on the current block to generate the prediction block of the current block.
  • the quantized coefficients obtained from the decoded code stream are inversely quantized and inversely transformed to obtain the residual block.
  • the prediction block and residual The blocks are added to obtain the reconstructed block, the reconstructed block constitutes the reconstructed image, and the reconstructed image is loop filtered based on the image or block to obtain the decoded image.
  • the encoding end also obtains the decoded image through similar operations as the decoding end.
  • the decoded image obtained by the encoding end is also usually called a reconstructed image.
  • the decoded image can be used as a reference frame for inter-frame prediction of subsequent frames.
  • the block division information determined by the encoding end, mode information and parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc. can be written into the code stream if necessary.
  • the decoding end determines the same block division information as the encoding end by decoding the code stream or analyzing the existing information, and determines the mode information and parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., thereby ensuring the decoding obtained by the encoding end.
  • the image is the same as the decoded image obtained at the decoding end.
  • block-based hybrid coding framework is used as an example above, the embodiments of the present disclosure are not limited thereto. With the development of technology, one or more modules in the framework, and one or more steps in the process Can be replaced or optimized.
  • the current block can be a block-level coding unit such as the current coding unit (CU) or the current prediction unit (PU) in the current image.
  • CU current coding unit
  • PU current prediction unit
  • the encoding end When the encoding end performs intra-frame prediction, it usually uses various angle modes and non-angle modes to predict the current block to obtain the prediction block; based on the rate distortion information calculated between the prediction block and the original block, the optimal prediction mode for the current block is screened out.
  • the prediction mode is encoded and transmitted to the decoder through the code stream.
  • the decoder obtains the prediction mode through decoding, and performs intra prediction on the current block according to the prediction mode.
  • non-angle modes have remained relatively stable, including mean mode (i.e. DC mode) and planar mode (i.e. Planar mode); angle modes have continued to increase with the evolution of digital video coding and decoding standards.
  • the H.264/AVC standard has only 8 traditional angle prediction modes and 1 traditional non-angle prediction mode; H.265/HEVC has been expanded to 33 traditional angle prediction modes and 2 traditional angle prediction modes.
  • the traditional intra prediction mode Planar mode, DC mode and 65 angle modes are shown in Figure 2.
  • the DC mode is suitable for large flat areas, and its prediction value is obtained by calculating the average value of the reference pixels on the left and/or upper side.
  • Planar mode is suitable for pixel gradients, that is, for areas where pixel values change slowly.
  • ECM Enhanced Compression Model
  • MPM technology first builds an MPM list, which is filled with the intra prediction mode most likely to be selected by the current block. If the prediction mode selected in the current block is in the MPM list, you only need to encode its index number (only 3 bits are needed). If the prediction mode selected in the current block is not in the MPM but in the 61 non-MPM (non-MPM) modes , then the truncated binary code (TBC) is used to encode the prediction mode in the entropy coding stage.
  • TBC truncated binary code
  • the MPM list has 6 prediction modes.
  • the MPM in ECM is divided into MPM and Secondary MPM (second MPM).
  • MPM and Secondary MPM are lists of length 6 and length 16 respectively.
  • the Planar mode is always at the first place in the MPM.
  • the remaining 5 positions are followed by the following three steps in sequence until the 5 MPM positions are filled.
  • the extra modes will automatically enter the Secondary MPM.
  • the prediction modes used by the five adjacent prediction blocks around the current block are shown in Figure 3, including the upper left (AL), upper (A), upper right (AR), left (L) and lower left (BL) of the current block.
  • the Secondary MP list consists of some primary angle modes other than those in the MPM.
  • Template based intra mode derivation is an intra prediction mode for luminance frames.
  • the TIMD mode is generated from the candidate modes and templates in the MPM list.
  • the left adjacent area and the upper adjacent area of the current block (shown as Current CU) 11 constitute the template (Template) area 12 of the current block, and the outside of the template area 12 (i.e. The left and upper sides) are provided with a template reference (reference of the template) area 13, and the exemplary size and position of each area are as shown in the figure.
  • the template reference area 13 may be an adjacent row above the template area or an adjacent column to the left.
  • TIMD assumes that the distribution characteristics of the current block and the template area of the current block are consistent, uses the reconstruction value of the template reference area as the reference line, traverses all modes in MPM and Secondary MPM to predict the template area, and obtains the prediction results. Then calculate the absolute sum of error transformations (SATD: Sum of absolute transformed differences) between the reconstructed value on the template area and the prediction result of each mode, select the mode with the smallest or optimal SATD, and use this mode as the current block TIMD mode.
  • the decoder can derive the TIMD mode through the same derivation method. If the sequence allows the use of TIMD, each current block requires a flag bit to indicate whether to use TIMD.
  • the intra prediction mode selected for the current block is the TIMD mode
  • the current block is predicted using the TIMD mode, and the decoding process of the remaining luma frames and intra prediction related syntax elements such as ISP and MPM can be skipped, thereby significantly Reduced mode encoding bits.
  • the final TIMD mode used can be determined according to the following method:
  • mode1 and mode2 are the two angle modes used for intra prediction in MPM.
  • mode1 is the angle mode with the smallest SATD, and its SATD is cost1;
  • mode2 is the angle mode with the second smallest SATD, and its SATD is cost2:
  • the prediction mode that weights the prediction results of mode1 and mode2 is used as the TMID mode of the current block, also called TIMD fusion mode.
  • the weighting method and weight are as follows:
  • Pred Pred mode1 ⁇ w1+Pred mode2 ⁇ w2
  • Pred is the prediction result of the current block using TIMD fusion mode
  • Pred mode1 is the prediction result of the current block using mode1
  • Pred mode2 is the prediction result of the current block using mode2
  • w1 and w2 are the weights calculated based on cost1 and cost2.
  • VVC In HEVC, intra prediction uses the previous row and left column closest to the current block as a reference for prediction. If the reconstruction value of this row and column has a large error with the original pixel value, then the prediction quality of the current block will also be affected. Great impact.
  • VVC adopts MRL intra prediction technology.
  • Reference line 0 Reference line 0
  • VVC can also use Reference line 1 (Reference line 1) and Reference line 2 ( Reference line2) is used as an extended reference line for intra prediction.
  • Reference line 1 Reference line 1
  • Reference line 2 Reference line2
  • MRL is only used in non-planar mode in MPM.
  • the decoding end When the encoding end predicts each angle mode, all three reference lines must be tried, and then the reference line with the smallest rate distortion cost (RD Cost) is selected, the index of the selected reference line and the selected intra prediction mode The same as the index, it also needs to be encoded and sent to the decoding end.
  • the decoding end decodes to obtain the index of the reference row, and then selects the corresponding reference row for prediction based on the index of the reference row.
  • Figure 5 shows an example of 4 reference lines, including reference line 0 (reference line0) 221 adjacent to the current block, that is, the reference line with index 0; reference line 1 (reference line 1) separated by 1 line from the current block. )222 is the reference line with index 1; reference line 2 (reference line2)223, which is 2 lines apart from the current block, is the reference line with index 2; and reference line 3 (reference line3)224, which is 3 lines apart from the current block That is the reference row with index 3. Only the reconstructed values of the reference row portion can be used during prediction.
  • the area in reference row 0 is regarded as the template area of the current block, then reference row 0 can be called the reference row where the template area is located, and reference rows 1 to 3 can be called the reference lines located outside the template area. Reference line.
  • indexes of reference rows are numbered in the manner shown in Figure 5. Except for the reference row with index 0, other reference rows are extended reference rows.
  • reference row is called a "row", this is for convenience of expression.
  • a reference row actually includes one row and one column.
  • the reconstructed value of the reference row used in prediction also includes the reconstructed value of one row and one column. This is consistent with The usual description method in the industry is the same.
  • Table 1 is used when the current block does not use TIMD
  • Table 2 is used when the current block uses TIMD.
  • the MRL tool can use more reference lines.
  • the indexes of multiple candidate reference lines are filled into a list.
  • this list is called a multi-reference row index list, abbreviated as an MRL index list, but it can also be called a multi-reference row list, a candidate reference row list, a reference row index list, etc.
  • the length of the multi-reference row index list is 6, that is, there are a total of 6 positions that can be filled with the indexes of 6 reference rows.
  • the index filled in the first position is 0, which is the index of the reference row closest to the current block.
  • the indexes filled in the second to sixth positions are 1, 3, and 5 respectively.
  • ,7,12 are the indexes of the 5 extended reference rows from the nearest to the farthest distance from the current block.
  • the multi-reference row index is used to indicate the position of the index of the reference row selected in the current block in the multi-reference row index list.
  • MultiRefIdx the index of the position of the reference row index in the multi-reference row index list is encoded.
  • MRL index list ⁇ 0,1,3,5,7,12 ⁇ as an example. The indexes from the 1st position to the 6th position are 0 to 5 respectively. If the index of the reference row selected in the current block is 7 , index 7 is at the fifth position in the multi-reference row index list, then the multi-reference row index is 4.
  • the multi-reference row index (multiRefIdx) can be encoded using a unary truncation code based on the context model, but is not limited to this.
  • MRL mode can also be used at the same time as TIMD mode.
  • the above example divides MRL into two tables when using TIMD and not using TIMD.
  • the above two tables can also be combined into one table.
  • the technique of deriving a list of most likely prediction modes (MPM) using blocks surrounding the current block may have different names, for example, in AV2 (AVM) it is called Adaptive Intra Mode Coding (AIMC). :Adaptive Intra Mode Coding), in AVS3, in the case of screen content encoding, it is called Frequency-based Intra Mode Coding (FIMC: Frequency-based Intra Mode Coding).
  • AIMC Adaptive Intra Mode Coding
  • FIMC Frequency-based Intra Mode Coding
  • the multi-reference row index list is composed of indexes in a specified fixed order.
  • the index of the selected reference row is at the front of the table, and the multi-reference code is The cost of row indexing is also smaller.
  • An embodiment of the present disclosure provides a multi-reference row index list sorting method, which can be used on both the encoding end and the decoding end. As shown in Figure 6, the method includes:
  • Step 110 Predict the template area respectively based on at least part of the reference lines of the current block located outside the template area, and calculate the error between the reconstructed value of the template area and the predicted value;
  • the above error can be expressed by absolute error and SAD, or by error transformed absolute value and SATD, but is not limited to this. It can also be expressed by sum of squared differences (SSD), mean absolute difference (MAD), mean squared error ( MSE) and so on.
  • Step 120 Fill in the MRL index list of the current block with the index of the reference row corresponding to the error in order of the error from small to large, to obtain an MRL index list sorted based on the current block.
  • the MRL index list is set by the encoding and decoding system, and the index and order of the reference lines filled in the MRL index list are fixed.
  • Embodiments of the present disclosure use a template-based approach to separately predict template areas based on at least part of the reference rows of the current block located outside the template area, and calculate the error between the reconstructed value of the template area and the predicted value. Fill in the MRL index list of the current block with the index of the reference row corresponding to the error in order of the largest to obtain the MRL index list sorted based on the current block.
  • what is constructed according to the embodiment of the present disclosure is a block-level MRL index list, in which the index and order of the reference rows are determined based on the distribution characteristics of the current block template area, so the reference rows selected in the current block are more likely to appear in the MRL index list , and is more likely to appear at the front of the MRL index list, that is, the value of the multi-reference row index that needs to be encoded is more likely to be a relatively small value, resulting in a smaller code length, less bandwidth usage, and faster decoding speed. , thereby improving encoding performance.
  • predicting the template area based on at least part of the reference lines of the current block located outside the template area includes: for each of the at least part of the reference lines, using the reference
  • the template region is predicted using the reconstruction value of the row and the intra prediction mode selected in the current block.
  • the intra prediction mode selection can be completed, and an intra prediction mode is selected for the current block (here refers to traditional intra prediction modes such as average mode, plane mode, and angle mode). Then, based on the intra prediction mode selected in the current block, the template area is predicted based on each reference row participating in the sorting.
  • the intra prediction related syntax elements of the current block can be decoded, and the intra prediction mode selected for the current block is determined, and then the above prediction can be performed to obtain the MRL index list sorted based on the current block. Afterwards, the reference row selected in the current block can be determined based on the multi-reference row index obtained by decoding.
  • filling the index of the reference row corresponding to the error into the MRL index list of the current block means filling the error corresponding to the K 1th position of the MRL index list.
  • the index of the reference row is filled in position by position, up to the K 2th position, where 2 ⁇ K 1 ⁇ K 2 ⁇ M, and M is the length of the MRL index list.
  • the actual index filled in the MRL index list can be less than or equal to M.
  • the indexes of these three reference rows are 1, 2, 3 respectively.
  • the reference row where the template area is located is not used for prediction. Therefore, the index of the reference row where the template area is located does not participate in the sorting, and these reference rows are more likely to be selected.
  • the method further includes: in the MRL index list sorted based on the current block, from the 1st position to the K 1 -1th position that does not participate in the sorting, fill in is the index of the reference row where the template area of the current block is located, where the index of the reference row filled in at the first position is 0.
  • the template area can also be extended, such as to the second row above the current block and the second column on the left, or to a larger area, that is, set in multiple reference rows closest to the current block. superior.
  • the indexes of more reference rows can be filled in the MRL index list.
  • the indexes of the reference rows that overlap with the template area must be filled in the MRL index list. You can also select part of them to fill in. MRL index list.
  • filling the index of the reference row corresponding to the error into the MRL index list of the current block means starting from the first position of the MRL index list.
  • the index of the reference row is filled in position by position, up to the K 2th position, where 1 ⁇ K 2 ⁇ M, and M is the length of the MRL index list.
  • this embodiment does not need to fill in the MRL index list with the index of the reference row where the template area of the current block is located.
  • the reference row selected by the current block When the row index is 0, that is, when the extended reference row is not used, the index of the selected reference row does not need to be represented by a multi-reference row index. It can be represented by setting the MRL usage flag (flag) to false. Therefore, the MRL index list does not need to be filled with index 0, and the indexes of the reference rows participating in the sorting can be filled in from the first position of the MRL index list in order from small to large errors.
  • K 2 ⁇ M starting from the K 2 +1-th position to the M-th position of the MRL index list, in order from closest to far from the current block Fill in the indexes of MK 2 reference rows position by position.
  • the MK 2 reference rows are located outside the template area and are specified by the system, and the indexes are different from the indexes that have been filled in the MRL index list.
  • the index of the reference row corresponding to the error is no longer filled in the order of the error from small to large, but the specified index outside the template area is filled in.
  • Reference line For example, the length M of the MRL index list is 6, and the index of the reference row sorted according to the error is only filled in the 2nd and 3rd positions.
  • the set MRL index list is ⁇ 0,1,3,5,7,12 ⁇ , and 1,3,5,7,12 are the indexes of the specified reference rows. This embodiment predicts based on the reference rows with indexes 1 to 12.
  • the MRL index list is ⁇ 0,5,7,1,3,12 ⁇ , that is, the indexes 1,3,12 of the reference row specified in the set MRL index list are filled into the rear of the sorted MRL index list. Location. If the two rows with the smallest error are the reference rows with indexes 4 and 6, since these two rows are not the specified reference rows, the sorted MRL index list is ⁇ 0,4,6,1,3,5 ⁇ , that is, the specified The index of the reference row is filled in position by position in order from the nearest to the farthest reference row from the current block.
  • the template area of the current block is set on a reference line closest to the current block, that is, it is set on the reference line with index 0:
  • the template area of the current block is set on On multiple reference lines closest to the current block; such as setting it on the two reference lines with indexes 0 and 1, or setting it on the three reference lines with indexes 0,1,2, or setting it on the index 0, On the four reference lines of 1, 2, 3, etc.
  • At least part of the reference lines of the current block located outside the template area include:
  • N reference lines are selected through error comparison between adjacent lines, N ⁇ N’;
  • N' ⁇ N max , N max is the maximum number of reference rows allowed to participate in sorting.
  • N' is equal to the number of reference lines between the outside of the template area and the upper boundary of the CTU.
  • the second and third methods are to select odd-numbered rows and even-numbered rows respectively from the reference rows determined in the first method to simplify the operation.
  • the fourth method of this embodiment is to select N reference lines from the N' reference lines outside the template area closest to the current block through error comparison between adjacent lines. This is achieved in the following way: using the reference line closest to the current block outside the template area as the first selected reference line, comparing the newly selected reference line with its adjacent reference lines one by one, and comparing the distance to the reference line. An adjacent reference row with the closest row and an error between the two reference rows greater than the set threshold is used as a newly selected reference row. This cycle continues until N' reference rows have been compared and selected.
  • the error refers to the error between the reconstructed values of two reference rows (the two reference rows participating in the comparison), or refers to the prediction obtained by predicting the template area using the reconstructed values of the two reference rows as a reference. error between values.
  • the reference lines outside the template area are numbered in order from closest to the current block.
  • the reference line closest to the current block is called the first outside the template area.
  • the reference line is called the first reference line for short;
  • the reference line next closest to the current block is called the second reference line outside the template area, which is called the second reference line for short; and so on. It is also assumed that the number of reference lines between the template area and the upper boundary of the CTU is greater than 12.
  • the comparison process is as follows: after selecting the first reference row outside the template area, compare the error between the first reference row and its adjacent row, that is, the second reference row. If the error is less than the set threshold, continue to compare the first reference row. The error between the reference row and the third reference row. If the error is greater than the set threshold, the third reference row is regarded as a newly selected row; then the third reference row is compared with its adjacent row (i.e.
  • the fourth reference row if the error is greater than the set threshold, the fourth reference row is used as a newly selected row; then the error of the fourth reference row and its adjacent row (i.e., the fifth reference row) is compared, By analogy, until the 12th reference row is compared, if the error between the 12th reference row and the most recently selected reference row is greater than the set threshold, the 12th reference row will be selected to participate in the sorting, otherwise Do not select the 12th reference row.
  • the MRL index list sorting method of the embodiment of the present disclosure is not limited to the sorting itself. It can also select the reference rows participating in the sorting, and then also select the reference rows to fill in the MRL index list with the index. . This helps find the optimal reference line and improves the efficiency of video encoding.
  • the present disclosure can also use specified reference lines for sorting.
  • the reference rows participating in sorting are the N reference rows specified from the N' reference rows closest to the current block outside the template area, for example, the specified indexes are 1,3,5,7,12 Reference line. At this time, it is equivalent to sorting the reference rows in the set MRL index list, and no new reference rows are introduced.
  • the N’ reference lines closest to the current block outside the template area are reference lines that do not exceed the upper boundary of the CTU of the coding tree unit where the current block is located;
  • N max set when the current block uses template-based intra mode to derive TIMD and N max set when the current block does not use TIMD are both greater than or equal to 3.
  • N max is set to 12 when the current block uses TIMD, and N max can also be set to 12 when the current block does not use TIMD.
  • This disclosure does not limit the specific value of N max . For example, it can be 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and values greater than 15, such as 18, 24, 36, etc. .
  • An embodiment of the present disclosure also provides a video decoding method, as shown in Figure 7, including:
  • Step 210 decode the intra prediction related syntax elements of the current block in the code stream, and continue to decode the multi-reference line index of the current block if the current block allows the use of multi-reference line MRL;
  • the multi-reference row index is used to indicate the position of the index of the reference row selected in the current block in the MRL index list.
  • Step 220 If it is determined according to the decoding result that sorting of the MRL index list of the current block is allowed, sort the MRL index list of the current block according to the method described in any embodiment of the present disclosure, and obtain an MRL index list sorted based on the current block. ;
  • Step 230 Determine the reference row selected for the current block based on the multi-reference row index and the MRL index list sorted based on the current block, and perform intra prediction on the current block based on the reference row selected for the current block.
  • the MRL index list of the current block is first sorted according to the method of the above embodiment of the present disclosure, and the MRL index sorted based on the current block is obtained. list; and then determine the reference row selected by the current block based on the decoded multi-reference row index and the MRL index list sorted based on the current block, and then perform intra prediction on the current block based on the reference row selected by the current block.
  • This embodiment can use the MRL index list based on the current block sorting to decode the multi-reference row index, so that the reference row selected in the current block is more likely to be located at the front of the MRL index list, thereby saving coding overhead and improving coding performance. .
  • the intra prediction mode selected by the current block is decoded for sorting the MRL index list of the current block; according to the current block
  • the current block is intra predicted based on the reference line selected on the current block and the intra prediction mode selected on the current block.
  • the method further includes: when it is determined according to the decoding result that sorting of the MRL index list of the current block is not allowed, determining according to the multi-reference line index and the set MRL index list The reference line selected in the current block is used to perform intra prediction on the current block based on the reference line selected in the current block.
  • the set MRL index list is used to determine the reference row pointed to by the multi-reference row index.
  • the set MRL index list when it is determined that the current block does not use TIMD, the set MRL index list is ⁇ 0,1,3,5,7,12 ⁇ ; when it is determined that the current block uses TIMD, The set MRL index list is ⁇ 0,1,3 ⁇ or ⁇ 0,1,2 ⁇ .
  • Whether the MRL index list of the current block is allowed to be sorted can be determined based on the decoded multi-reference row index, or it can be expressed directly using a flag.
  • the method further includes: when it is determined that the MRL index list of the current block is allowed to be sorted, setting a flag of the current block indicating that the MRL index list is allowed to be sorted to true.
  • any one or more of the following conditions are used as sufficient conditions to disallow sorting the MRL index list of the current block:
  • the current block does not use multi-reference line MRL;
  • the multi-reference row index indicates the position in the MRL index list that does not participate in sorting, such as the first position
  • the current frame is not a brightness frame
  • the current block is located at the upper boundary of the coding tree unit CTU;
  • the intra prediction mode selected in the current block is a designated mode, and the designated mode includes any one or more of the following modes: planar mode, TIMD fusion mode with one mode being planar mode, average mode, horizontal angle mode, Vertical angle mode, wide angle mode;
  • the size of the current block does not meet the set conditions
  • the aspect ratio of the current block does not meet the set conditions.
  • one of the current block using TIMD and the current block not using TIMD can also be used as a sufficient condition for not allowing the MRL index list of the current block to be sorted. That is, at this time, the MRL index list is only allowed to be sorted when the current block uses TIMD or the current block does not use TIMD.
  • the method when the current block uses MRL, the method further includes: decoding syntax elements of some transformation modes that the current block can use simultaneously with MRL, and skipping those that cannot be used simultaneously with MRL. Syntax elements for partial transformation patterns.
  • This embodiment can skip some transformation modes when the current block uses MRL.
  • the MRL mode is in a relatively low position among all intra prediction modes. For example, it is necessary to determine whether the current mode is not DIMD, BDPCM, MIP, or TIMD before determining whether it is MRL.
  • the residual signal can use primary transformation and secondary transformation to make the transformed residual easier to encode.
  • One of the transformations includes DCT2 transformation and DST7.
  • the secondary transformation is to further transform the residual after the primary transformation on the basis of using DCT2 in the primary transformation.
  • ECM the mode of one-time transformation has also been further expanded. Considering that when different modes are selected, extra bits are required to represent them, which will bring codeword overhead.
  • the method when the current block uses MRL, the method further includes: in the process of decoding the intra prediction mode selected by the current block, allowing decoding of syntax elements related to the second MPM .
  • This embodiment allows MRL and second MPM modes to be used simultaneously.
  • decoding the multi-reference line index of the current block includes: decoding the current block according to different MRL index lists when the current block uses TIMD and when TIMD is not used. Multiple reference row index.
  • An embodiment of the present disclosure also provides a video encoding method, as shown in Figure 8, including:
  • Step 310 Perform intra prediction on the current block and determine the reference line selected in the current block;
  • Step 320 When it is determined that sorting of the MRL index list of the current block is allowed, sort the MRL index list of the current block according to the method as described in any one of claims 1 to 11, and obtain an MRL index list sorted based on the current block. ;
  • Step 330 Determine and encode the multi-reference row index of the current block based on the position of the index of the reference row selected in the current block in the MRL index list sorted based on the current block.
  • the MRL index list of the current block is first sorted according to the method of the above embodiment of the present disclosure, and the MRL index sorted based on the current block is obtained. list; and then determine and encode the multi-reference row index of the current block based on the position of the index of the reference row selected in the current block in the MRL index list sorted based on the current block.
  • the MRL index list based on the current block sorting can be used to encode the multi-reference row index, so that the reference row selected in the current block is more likely to be located at the front of the MRL index list, thereby saving the coding overhead of the multi-reference row index. , improve encoding performance.
  • the method before sorting the MRL index list of the current block, the method further includes: determining the intra prediction mode selected for the current block.
  • the method further includes: when determining that the MRL index list of the current block is not allowed to be sorted, the index of the reference row selected according to the current block is in the set MRL index list. Position, determine the multi-reference row index of the current block and encode it. In this embodiment, when it is determined that sorting of the MRL index list of the current block is not allowed, the position of the selected reference row in the set MRL index list is used to determine the multi-reference row index of the current block.
  • the method further includes: when it is determined that the MRL index list of the current block is allowed to be sorted, setting a flag of the current block indicating that the MRL index list is allowed to be sorted to true.
  • a flag is directly used to indicate whether sorting of the MRL index list of the current block is allowed.
  • any one or more of the following conditions are regarded as sufficient conditions for not allowing the MRL index list of the current block to be sorted:
  • the current block does not use MRL
  • the reference line selected in the current block is located in the template area of the current block
  • the current frame is not a brightness frame
  • the current block is located at the upper boundary of the coding tree unit CTU;
  • the intra prediction mode selected in the current block is a designated mode, and the designated mode includes any one or more of the following modes: planar mode, TIMD fusion mode with one mode being planar mode, average mode, horizontal angle mode, Vertical angle mode, wide angle mode;
  • the size of the current block does not meet the set conditions
  • the aspect ratio of the current block does not meet the set conditions
  • One of the current block using TIMD and the current block not using TIMD is
  • different MRL index lists are used when the current block uses TIMD and when TIMD is not used.
  • Determining and encoding the multi-reference row index of the current block includes:
  • the multi-reference row index of the current block is determined and encoded according to the position of the index of the reference row selected in the current block in the first MRL index list;
  • the length of the second MRL index list is greater than or equal to the length of the first MRL index list.
  • the method when the current block uses MRL, the method further includes: encoding syntax elements of some transformation modes that the current block can use simultaneously with MRL, and skipping those that cannot be used simultaneously with MRL. Encoding of partial transformation modes.
  • the method when the current block uses MRL, and the selected mode of the current block is not the TIMD mode.
  • the method also includes: after encoding the multi-reference row index, continue to encode the intra prediction mode selected by the current block, and in the process of encoding the intra prediction mode selected by the current block, allow encoding of syntax related to the second MPM element.
  • An embodiment of the present disclosure also provides a code stream, wherein the code stream includes a multi-reference line index of the current block, and the multi-reference line index is used to indicate that the index of the reference line selected in the current block is in the MRL index list. position in, the MRL index list is the MRL index list sorted based on the current block, or the set MRL index list.
  • the code stream in this embodiment can be generated by the video encoding method described in any embodiment of this disclosure.
  • An embodiment of the present disclosure also provides a multi-reference row index list sorting device, as shown in Figure 9, including a processor 71 and a memory 73 storing a computer program, wherein when the processor 71 executes the computer program
  • a multi-reference row index list sorting device as shown in Figure 9, including a processor 71 and a memory 73 storing a computer program, wherein when the processor 71 executes the computer program
  • the multi-reference row index list sorting method described in any embodiment of the present disclosure can be implemented.
  • An embodiment of the present disclosure also provides a video decoding device, see FIG. 9 , including a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement any embodiment of the present disclosure.
  • the video decoding method see FIG. 9 , including a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a video encoding device, see FIG. 9 , including a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement any embodiment of the present disclosure.
  • the video encoding method includes a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement any embodiment of the present disclosure.
  • the processor in the above embodiments of the present disclosure may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), a microprocessor, etc., or it may be other conventional processors. Processor, etc.; the processor may also be a digital signal processor (DSP), application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gates or transistor logic devices , discrete hardware components; it can also be a combination of the above devices. That is, the processor in the above embodiments can be any processing device or device combination that implements the methods, steps and logical block diagrams disclosed in the embodiments of the present invention.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • instructions for the software may be stored in a suitable non-volatile computer-readable storage medium and may be executed in hardware using one or more processors. Instructions are provided to perform the methods of embodiments of the present disclosure.
  • An embodiment of the present disclosure also provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements any implementation of the present disclosure.
  • An embodiment of the present disclosure provides a video decoding method, which sorts the multi-reference line index list of the current block during the decoding process.
  • the sorting based on the template area in this embodiment is an operation that both the encoder and the decoder need to perform. It acts on the MRL index list and represents the position of the selected reference line in the MRL index list through the multi-reference line index. Combined with the current
  • the block-sorted MRL index list jointly determines the reference line selected for the current block, and then the determined reference line and the selected intra prediction mode (traditional prediction mode) are used for prediction.
  • This embodiment uses a specific example to illustrate how to use the sum of absolute errors (SAD: the sum of absolute difference) of the predicted value (pred) and the reconstructed value (reco) on the template area to sort the MRL index list.
  • SAD the sum of absolute difference
  • (x,-1), (-1,y) are the coordinates relative to the upper left corner (0,0) of the current block respectively.
  • the template area of 1 row and 1 column of the current block is shown in Figure 10 The hatched area shown. This figure shows six reference lines outside the current block template area.
  • the decoder parses the intra prediction mode, confirms that the MRL-related syntax elements parsed by the decoder are used or not using MRL on the current block, and decodes the multi-reference line index of the current block.
  • intra_luma_mrl_tab_idx defines the multi-reference row index when the current block does not use TIMD (ie! intra_timd_flag) to indicate the position of the reference row index selected in the current block in the sorted MRL index list, and its value The range is 0 to 5.
  • intra_timd_luma_ref_idx defines the multi-reference row index when the current block uses TIMD to indicate the position of the index of the reference row selected in the current block in the set MRL index list. Its value range is 0 to 2.
  • the multi-reference row index in both cases can be encoded and decoded using truncated unary codes based on the context model.
  • This example does not sort the set MRL index list when the current block uses TIMD. But in other examples, in the case where the current block uses TIMD, it is also possible to generate an MRL index list sorting for the current block.
  • sorting the MRL index list when the current block uses TIMD you can only sort the reference rows in the MRL index list set by the index filling, or you can introduce more reference rows, for example, the current block does not use TIMD. case the same reference row participates in the sorting.
  • a syntax element similar to intra_luma_mrl_tab_idx can be used instead of intra_timd_luma_ref_idx.
  • the timd in the table of this article is the same concept as the TIMD in the text, and other concepts are similar.
  • MRL index list sorting of blocks Sorting is only performed on extended reference rows (in this embodiment, reference rows whose index is not 0).
  • the sorting process is as follows.
  • the upper reference lines that can be used by the current block should not exceed the upper CTU boundary.
  • the CTU can be up to 256x256.
  • the resulting remainder is the number of reference lines above within the CTU boundary.
  • the reference lines used may also exceed CTU boundaries if supported by the hardware.
  • the N reference lines reference line 1 to N are outside the template area of the line where reference line 0 is located.
  • the selected intra prediction mode is used to predict the template area. Since the template area of reference line 0 has been reconstructed, the SAD can be calculated based on the reconstructed value and the predicted value. Sort the N SAD values in ascending order, and fill the MRL index list with the indexes of the M reference rows corresponding to the smallest calculated SAD.
  • the length of the MRL index list is 6 when the current block does not use TIMD, and the length is 3 when TIMD is used, and the padding values set for these two lists are ⁇ 0,1,3,5, 7,12 ⁇ and ⁇ 0,1,3 ⁇ represent the reference rows with indexes 0,1,3,5,7,12 and the reference rows with indexes 0,1,3 respectively.
  • the sorting in this embodiment is to sort reference lines 1 ⁇ N, so the index 0 filled in the first position in the MRL index list will always remain unchanged, and only the sorted extended reference line will be filled in the subsequent positions. In the case where the current block does not use TIMD, up to 5 indexes can be filled in; in the case where the current block uses TIMD, up to 2 indexes can be filled in.
  • Method 1 Calculate the SAD value between the reconstructed values at the corresponding positions of two adjacent reference rows. When the SAD value is less than a preset threshold, it is considered that the two reference rows are too similar and one of the reference rows should be skipped. If the adjacent reference line is skipped, the selected reference line continues to be threshold compared with the next nearest reference line.
  • Method 2 Use two reference lines to predict the template area, and calculate the SAD value between the prediction results in the template area. When the SAD value is less than a preset threshold, it is considered that the two lines are too similar and should be skipped. One line.
  • the above skipping method can skip any one of the two rows, but considering that usually the reference row closer to the current block is more suitable as a reference row, this embodiment skips the farther reference row.
  • the intra prediction mode decided is the TIMD fusion mode, and the combination of MRL and Planar mode usually does not bring additional performance improvement, so when MRL When selected, if the intra prediction mode is Planar or one of the TIMD fusion (fusion) modes is Planar, there is no need to sort. In addition to Planar mode, you can also not sort in some specific angles or modes, such as not sorting in DC mode, not sorting in wide angle, not sorting in horizontal and vertical angle modes, etc. The wide-angle mode can be equivalent to not sorting rectangular prediction blocks.
  • the initial reference row lists such as ⁇ 1,3,5,7,12 ⁇ are directly used without reordering the reference rows in these lists based on SAD.
  • This embodiment can skip some transformation modes.
  • the current MRL encoding is at a relatively low position among all intra-frame modes. In ECM, it is necessary to determine whether the current mode is not DIMD, BDPCM, MIP, or TIMD before determining whether it is MRL. .
  • MRL can also be restricted to only be used in combination with some transformation modes, for example, it can only be used with DCT2 Can be used with or only with the primary and secondary transforms of DCT2. For example, when restricting MRL to be used together with DCT2 under one transformation, the syntax element of the decoding transformation mode should check whether MRL has been selected. If it is selected, the one transformation mode will not be decoded.
  • the syntax element of the decoding transformation mode should check whether MRL has been selected, and if it is selected, the secondary transformation mode will not be decoded.
  • the corresponding syntax is as follows:
  • the reference row is determined based on the reordered multi-reference row index list and the decoded list index and the current block is predicted.
  • the error between the predicted value and the reconstructed value of the template area is used to sort in ascending order to obtain a multi-reference row index list based on the current block sorting, thereby making it more accurate. It is possible that the reference row selected for prediction is ranked higher in the multi-reference row index list, saving coding overhead.
  • threshold settings are used to skip some possibly repeated reference rows to be sorted, which can reduce redundancy and improve sorting efficiency.
  • the reference row where the template area is located and the reference rows smaller than the reference row cannot be added to the sorting, and their original positions in the candidate list can be maintained.
  • sortable reference rows can be added.
  • this embodiment uses up to 12 extended reference rows to participate in sorting. In fact, without considering the complexity, this restriction can be removed so that more than 12 reference lines can be used for sorting, thereby obtaining better encoding and decoding performance.
  • the size of the template area can be increased.
  • the template area is not limited to the reference line closest to the current block, but can continue to expand, such as extending to the right of the upper template area, below the left template area, and to the gray intersection in the upper left corner. Partially extended. You can also continue to expand above the upper template area and to the left side of the left template area. This will use more reference line areas. For example, when the reference line 1 area is further used as the template area, reference line 1 itself cannot Participate in sorting.
  • the above test also uses reference lines 1 to 12 to participate in sorting when the current block uses TIMD, and only retains the two reference lines line_a and line_b with the smallest SAD to fill in the sorted MRL index list: ⁇ 0, line_a, line_b ⁇ .
  • Encoding Time, encoding time, 10X% means that when the reference row sorting technology is integrated, the encoding time is 10X% compared to before it is not integrated, which means that there is an X% increase in encoding time.
  • DecT Decoding Time, decoding time, 10X% means that when the reference row sorting technology is integrated, the decoding time is 10X% compared to before it is not integrated, which means that there is an X% increase in decoding time.
  • ClassA1 and Class A2 are test video sequences with a resolution of 3840x2160
  • ClassB is a test sequence with a resolution of 1920x1080
  • ClassC is 832x480
  • ClassD is 416x240
  • ClassE is 1280x720
  • ClassF is a screen content sequence of several different resolutions (Screen content) .
  • All intra represents the test configuration of the full intra frame configuration.
  • Computer-readable media may include computer-readable storage media that corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, such as according to a communications protocol.
  • Computer-readable media generally may correspond to non-transitory, tangible computer-readable storage media or communication media such as a signal or carrier wave.
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in this disclosure.
  • a computer program product may include computer-readable media.
  • Such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory or may be used to store instructions or data. Any other medium that stores the desired program code in the form of a structure and that can be accessed by a computer.
  • any connection is also termed a computer-readable medium if, for example, a connection is sent from a website, server, or using any of the following: coaxial cable, fiber-optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, and microwave or other remote source transmits instructions, then coaxial cable, fiber optic cable, twin-wire, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium.
  • coaxial cable, fiber optic cable, twin-wire, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium.
  • disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, or Blu-ray discs. Disks usually reproduce data magnetically, while optical discs use lasers to reproduce data. Regenerate data optically. Combinations of the above should also be included within the scope of computer-readable media.
  • processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • Execute instructions may refer to any of the structures described above or any other structure suitable for implementing the techniques described herein.
  • the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec.
  • the techniques may be implemented entirely in one or more circuits or logic elements.
  • inventions of the present disclosure may be implemented in a wide variety of devices or equipment, including wireless handsets, integrated circuits (ICs), or a set of ICs (eg, chipsets).
  • ICs integrated circuits
  • a set of ICs eg, chipsets.
  • Various components, modules or units are depicted in embodiments of the present disclosure to emphasize functional aspects of devices configured to perform the described techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperating hardware units (including one or more processors as described above) in conjunction with suitable software and/or firmware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种多参考行索引列表排序方法、视频编解码方法、装置和系统，根据当前块位于模板区域外侧的至少部分参考行对所述模板区域分别进行预测，计算所述模板区域的重建值和预测得到的预测值之间的误差；按照所述误差从小到大的顺序，将所述误差对应的参考行的索引填入当前块的MRL索引列表，得到基于当前块排序后的MRL索引列表；在编解码时，可对当前块的MRL索引列表排序，根据排序后的MRL索引列表来解码或编码当前块的多参考行索引。本公开实施例可以节约编码开销，提升编码性能。

Description

多参考行索引列表排序方法、视频编解码方法、装置和系统 技术领域
本公开实施例涉及但不限于视频技术，更具体地，涉及一种多参考行索引列表排序方法、视频编解码方法、装置和系统。
背景技术
数字视频压缩技术主要是将庞大的数字影像视频数据进行压缩，以便于传输以及存储等。目前通用的视频编解码标准，如H.266/Versatile Video Coding(多功能视频编码，VVC)，都采用基于块的混合编码框架。视频中的每一帧被分割成相同大小(如128x128，64x64等)的正方形的最大编码单元(LCU：largest coding unit)。每个最大编码单元可根据规则划分成矩形的编码单元(CU：coding unit)。编码单元可能还会划分预测单元(PU：prediction unit)，变换单元(TU：transform unit)等。混合编码框架包括预测(prediction)、变换(transform)、量化(quantization)、熵编码(entropy coding)、环路滤波(in loop filter)等模块。预测模块包括用于减少或取出视频内在冗余的帧内预测(intra prediction)和帧间预测(inter prediction)。帧内块通过块周边像素作为参考进行预测，帧间块则参考空间上的邻近块信息和其他帧里的参考信息。与预测信号相对，残差信息通过以块为单位的变换、量化和熵编码成码流。这些技术被描述在标准里并实施在的各种与视频压缩相关的领域。
随着互联网视频的激增以及人们对视频清晰度的要求越来越高，尽管已有的数字视频压缩标准能够节省不少视频数据，但目前仍然需要追求更好的数字视频压缩技术，以减少数字视频传输的带宽和流量压力。
发明概述
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本公开一实施例提供了一种多参考行索引列表排序方法，包括：
根据当前块位于模板区域外侧的至少部分参考行对所述模板区域分别进行预测，计算所述模板区域的重建值和预测得到的预测值之间的误差；
按照所述误差从小到大的顺序，将所述误差对应的参考行的索引填入当前块的MRL索引列表，得到基于当前块排序后的MRL索引列表。
本公开一实施例还提供了一种视频解码方法，包括：
解码码流中当前块的帧内预测相关语法元素，在当前块允许使用多参考行MRL的情况下，继续解码当前块的多参考行索引；所述多参考行索引用于指示当前块选中的参考行的索引在MRL索引列表中的位置；
根据解码结果确定允许对当前块的MRL索引列表排序的情况下，按照如本公开任一实施例所述的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；
根据所述多参考行索引和基于当前块排序后的MRL索引列表，确定当前块选中的参考行，根据当前块选中的参考行对当前块进行帧内预测。
本公开一实施例还提供了一种视频编码方法，包括：
对当前块进行帧内预测，确定当前块选中的参考行；
确定允许对当前块的MRL索引列表排序的情况下，按照如本公开任一实施例所述的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；
根据当前块选中的参考行的索引在基于当前块排序后的MRL索引列表中的位置，确定当前块的多参考行索引并编码。
本公开一实施例还提供了一种码流，其中，所述码流中包括当前块的多参考行索引，所述多参考行索引用于指示当前块选中的参考行的索引在MRL索引列表中的位置，所述MRL索引列表是基于当前块排序后的MRL索引列表，或者是设定的MRL索引列表。
本公开一实施例还提供了一种多参考行索引列表排序装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的多参考行索引 列表排序方法。
本公开一实施例还提供了一种视频解码装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的视频解码方法。
本公开一实施例还提供了一种视频编码装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的视频编码方法。
本公开一实施例还提供了一种视频编解码系统，其中，包括本公开任一实施例所述的视频编码装置和本公开任一实施例所述的视频解码装置。
本公开一实施例还提供了一种非瞬态计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其中，所述计算机程序时被处理器执行时实现如本公开任一实施例所述的多参考行索引列表排序方法，或实现本公开任一实施例所述的视频解码方法，或实现本公开任一实施例所述的视频编码方法。
在阅读并理解了附图和详细描述后，可以明白其他方面。
附图概述
附图用来提供对本公开实施例的理解，并且构成说明书的一部分，与本公开实施例一起用于解释本公开的技术方案，并不构成对本公开技术方案的限制。
图1A是本公开一实施例编解码系统的示意图；
图1B是本公开一实施例编码端的框架图；
图1C是本公开一实施例解码端的框架图；
图2是本公开一实施例帧内预测模式的示意图；
图3是本公开一实施例当前块的相邻帧内预测块的示意图；
图4是本公开一实施例当前块的模板区域及模板参考区域的示意图；
图5是本公开一实施例当前块周围的多个参考行的示意图；
图6是本公开一实施例多参考行索引列表排序方法的流程图；
图7是本公开一实施例视频解码方法的流程图；
图8是本公开一实施例视频编码方法的流程图；
图9是本公开一实施例多参考行索引列表排序装置的示意图；
图10是本公开一实施例当前块周围的模板及多个参考行的示意图。
详述
本公开描述了多个实施例，但是该描述是示例性的，而不是限制性的，并且对于本邻域的普通技术人员来说显而易见的是，在本公开所描述的实施例包含的范围内可以有更多的实施例和实现方案。
本公开的描述中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本公开中被描述为“示例性的”或者“例如”的任何实施例不应被解释为比其他实施例更优选或更具优势。本文中的“和/或”是对关联对象的关联关系的一种描述，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。“多个”是指两个或多于两个。另外，为了便于清楚描述本公开实施例的技术方案，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本邻域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
在描述具有代表性的示例性实施例时，说明书可能已经将方法和/或过程呈现为特定的步骤序列。然而，在该方法或过程不依赖于本文所述步骤的特定顺序的程度上，该方法或过程不应限于所述的特定顺序的步骤。如本邻域普通技术人员将理解的，其它的步骤顺序也是可能的。因此，说明书中阐述的步骤的特定顺序不应被解释为对权利要求的限制。此外，针对该方法和/或过程的权利要求不应限于按照所写顺序执行它们的步骤，本邻域技术人员可以容易地理解，这些顺序可以变化，并且仍然保持在本公开实施例的精神和范围内。
本公开实施例提出的局部光照补偿方法、视频编解码方法可以应用于各种视频编解码标准，例如： H.264/Advanced Video Coding(高级视频编码，AVC),H.265/High Efficiency Video Coding(高效视频编码，HEVC),H.266/Versatile Video Coding(多功能视频编码，VVC)，AVS(Audio Video coding Standard，音视频编码标准)，以及MPEG(Moving Picture Experts Group，动态图像专家组)、AOM(开放媒体联盟，Alliance for Open Media)、JVET(联合视频专家组,Joint Video Experts Team)制订的其他标准以及这些标准的拓展，或任何自定义的其他标准等。
图1A是可用于本公开实施例的一种视频编解码系统的框图。如图所示，该系统分为编码端装置1和解码端装置2，编码端装置1产生码流。解码端装置2可对码流进行解码。解码端装置2可经由链路3从编码端装置1接收码流。链路3包括能够将码流从编码端装置1移动到解码端装置2的一个或多个媒体或装置。在一个示例中，链路3包括使得编码端装置1能够将码流直接发送到解码端装置2的一个或多个通信媒体。编码端装置1根据通信标准(例如无线通信协议)来调制码流，将经调制的码流发送到解码端装置2。所述一个或多个通信媒体可包含无线和/或有线通信媒体,可形成分组网络的一部分。在另一示例中，也可将码流从输出接口15输出到一个存储装置，解码端装置2可经由流式传输或下载从该存储装置读取所存储的数据。
如图所示，码端装置1包含数据源11、视频编码装置13和输出接口15。数据源11包括视频捕获装置(例如摄像机)、含有先前捕获的数据的存档、用以从内容提供者接收数据的馈入接口，用于产生数据的计算机图形系统，或这些来源的组合。视频编码装置13对来自数据源11的数据进行编码后输出到输出接口15，输出接口15可包含调节器、调制解调器和发射器中的至少之一。解码端装置2包含输入接口21、视频解码装置23和显示装置25。输入接口21包含接收器和调制解调器中的至少之一。输入接口21可经由链路3或从存储装置接收码流。视频解码装置23对接收的码流进行解码。显示装置25用于显示解码后的数据，显示装置25可以与解码端装置2的其他装置集成在一起或者单独设置，显示装置25对于解码端来说是可选的。在其他示例中，解码端可以包含应用解码后数据的其他装置或设备。
基于图1A所示的视频编解码系统，可以使用各种视频编解码方法来实现视频压缩和解压缩。
图1B为可用于本公开实施例的一示例性的视频编码装置的框图。如图所示，该视频编码装置1000包含预测单元1100、划分单元1101、残差产生单元1102(图中用划分单元1101后的带加号的圆圈表示)、变换处理单元1104、量化单元1106、反量化单元1108、反变换处理单元1110、重建单元1112(图中用反变换处理单元1110后的带加号的圆圈表示)、滤波器单元1113、解码图像缓冲器1114以及熵编码单元1115。其中，预测单元1100包含帧间预测单元1121和帧内预测单元1126，解码图像缓冲器1114也可以称为已解码图像缓冲器、解码图片缓冲器、已解码图片缓冲器等。视频编码器20也可以包含比该示例更多、更少或不同功能组件，如在某些情况下可以取消变换处理单元1104、反变换处理单元1110等。
划分单元1101与预测单元1100配合将接收的视频数据划分为切片(Slice)、编码树单元(CTU：Coding Tree Unit)或其它较大的单元。划分单元1101接收的视频数据可以是包括I帧、P帧或B帧等视频帧的视频序列。
预测单元1100可以将CTU划分为编码单元(CU：Coding Unit)，对CU执行帧内预测编码或帧间预测编码。对CU做帧内预测和帧间预测时，可以将CU划分为一个或多个预测单元(PU：prediction unit)。
帧间预测单元1121可对PU执行帧间预测，产生PU的预测数据，所述预测数据包括PU的预测块、PU的运动信息和各种语法元素。帧间预测单元1121可以包括运动估计(ME:motion estimation)单元和运动补偿(MC:motion compensation)单元。运动估计单元可以用于运动估计以产生运动矢量，运动补偿单元可以用于根据运动矢量获得或生成预测块。
帧内预测单元1126可对PU执行帧内预测，产生PU的预测数据。PU的预测数据可包含PU的预测块和各种语法元素。
残差产生单元1102可基于CU的原始块减去CU划分成的PU的预测块，产生CU的残差块。
变换处理单元1104可将CU划分为一个或多个变换单元(TU：Transform Unit)，预测单元和变换单元的划分可以不同。TU关联的残差块是CU的残差块划分得到的子块。通过将一种或多种变换应用于TU关联的残差块来产生TU关联的系数块。
量化单元1106可基于选定的量化参数对系数块中的系数进行量化，通过调整量化参数(QP：Quantizer Parameter)可以调整对系数块的量化程度。
反量化单元1108和反变换单元1110可分别将反量化和反变换应用于系数块，得到TU关联的重建残差块。
重建单元1112可将所述重建残差块和预测单元1100产生的预测块相加，产生重建图像。
滤波器单元1113对重建图像执行环路滤波，将滤波后的重建图像存储在解码图像缓冲器1114中作为参考图像。帧内预测单元1126可以从解码图像缓冲器1114中提取PU邻近的块的参考图像以执行帧内预测。帧间预测单元1121可使用解码图像缓冲器1114缓存的上一帧的参考图像对当前帧图像的PU执行帧间预测。
熵编码单元1115可以对接收的数据(如语法元素、量化后的系数块、运动信息等)执行熵编码操作。
图1C为可用于本公开实施例的一示例性的视频解码装置的框图。如图所示，视频解码装置101包含熵解码单元150、预测单元152、反量化单元154、反变换处理单元156、重建单元158(图中用反变换处理单元155后的带加号的圆圈表示)、滤波器单元159，以及解码图像缓冲器160。在其它实施例中，视频解码器30可以包含更多、更少或不同的功能组件，如在某些情况下可以取消反变换处理单元155等。
熵解码单元150可对接收的码流进行熵解码，提取语法元素、量化后的系数块和PU的运动信息等。预测单元152、反量化单元154、反变换处理单元156、重建单元158以及滤波器单元159均可基于从码流提取的语法元素来执行相应的操作。
反量化单元154可对量化后的TU关联的系数块进行反量化。
反变换处理单元156可将一种或多种反变换应用于反量化后的系数块以便产生TU的重建残差块。
预测单元152包含帧间预测单元162和帧内预测单元164。如果PU使用帧内预测编码，帧内预测单元164可基于从码流解码出的语法元素确定PU的帧内预测模式，根据确定的帧内预测模式和从解码图像缓冲器160获取的PU邻近的已重建参考信息执行帧内预测，产生PU的预测块。如果PU使用帧间预测编码，帧间预测单元162可基于PU的运动信息和相应的语法元素来确定PU的一个或多个参考块，基于从解码图像缓冲器160获取的所述参考块来产生PU的预测块。
重建单元158可基于TU关联的重建残差块和预测单元152产生的PU的预测块，得到重建图像。
滤波器单元159可对重建图像执行环路滤波，滤波后的重建图像存储在解码图像缓冲器160中。解码图像缓冲器160可提供参考图像以用于后续运动补偿、帧内预测、帧间预测等，也可将滤波后的重建图像作为已解码视频数据输出，在显示装置上的呈现。
基于上述视频编码装置和视频解码装置，可以执行以下基本的编解码流程，在编码端，将一帧图像划分成块，对当前块进行帧内预测或帧间预测或其他算法产生当前块的预测块，使用当前块的原始块减去预测块得到残差块，对残差块进行变换和量化得到量化系数，对量化系数进行熵编码生成码流。在解码端，对当前块进行帧内预测或帧间预测产生当前块的预测块，另一方面对解码码流得到的量化系数进行反量化、反变换得到残差块，将预测块和残差块相加得到重建块，重建块组成重建图像，基于图像或基于块对重建图像进行环路滤波得到解码图像。编码端同样通过和解码端类似的操作以获得解码图像，编码端获得的解码图像通常也叫做重建图像。解码图像可以作为对后续帧进行帧间预测的参考帧。编码端确定的块划分信息，预测、变换、量化、熵编码、环路滤波等模式信息和参数信息如果需要可以写入码流。解码端通过解码码流或根据已有信息进行分析，确定与编码端相同的块划分信息，预测、变换、量化、熵编码、环路滤波等模式信息和参数信息，从而保证编码端获得的解码图像和解码端获得的解码图像相同。
以上虽然是以基于块的混合编码框架为示例，但本公开实施例并不局限于此，随着技术的发展，该框架中的一个或多个模块，及该流程中的一个或多个步骤可以被替换或优化。
本文中，当前块(current block)可以是当前图像中的当前编码单元(CU)、当前预测单元(PU)等块级编码单位。
编码端在帧内预测时，通常借助各种角度模式与非角度模式对当前块进行预测得到预测块；根据 预测块与原始块计算得到的率失真信息，筛选出当前块最优的预测模式，将该预测模式编码后经码流传输到解码端。解码端通过解码得到该预测模式，按照该预测模式对当前块进行帧内预测。经过历代的数字视频编解码标准发展，非角度模式保持相对稳定，有均值模式(即DC模式)和平面模式(即Planar模式)；角度模式则随着数字视频编解码标准的演进而不断增加，以国际数字视频编码标准H系列为例，H.264/AVC标准仅有8种传统角度预测模式和1种传统非角度预测模式；H.265/HEVC扩展到33种传统角度预测模式和2种传统非角度预测模式。在H.266/VVC中，传统帧内预测模式Planar模式，DC模式和65种角度模式，如图2所示。其中，DC模式适用于大面积平坦区域，其预测值通过计算左边和/或上边的参考像素的平均值获得。Planar平面模式适用于像素渐变的情况，即适用于像素值缓慢变化的区域。
如果直接对预测块的模式进行编码，67种模式需要7bit来编码，数据量很大。根据统计特性，距离当前块越近的像素区域往往越容易与当前块选中同样的帧内预测模式，根据这一特性，在HEVC、VVC以及增强的压缩模型(ECM：Enhanced Compression Model)中都采纳了最可能模式(MPM：most probable mode)技术。ECM是基于VTM-10.0参考软件，集成各类新工具，从而进一步挖掘编解码性能的参考软件。
MPM技术先构建MPM列表，MPM列表中填充最有可能被当前块选中的帧内预测模式。如果当前块选中的预测模式在MPM列表中，只需要编码其索引号(只需要3bit)即可，如果当前块选中的预测模式不在MPM中而是在61个非MPM(non-MPM)模式中，则在熵编码阶段使用截断二元码(Truncated Binary Code,TBC)编码该预测模式。
在VVC内，无论是否应用多参考行(MRL：Multiple reference line)和帧内子划分(ISP：Intra Sub-Partitions)，MPM list均有6种预测模式。而ECM中的MPM分为MPM和Secondary MPM(第二MPM)，MPM和Secondary MPM分别是长度为6与长度为16的列表。MPM列表的6个模式中，Planar模式始终在MPM的第一位，其余5个位置由下面三步依次进行，直到填充满5个MPM位置为止，多出来的模式将自动进入Secondary MPM。第一步，当前块周围临近的5个预测块使用的预测模式，如图3所示，包括当前块左上(AL)，上(A)，右上(AR)，左(L)及左下(BL)位置上的预测块使用的预测模式；第二步，当前块周围重建像素使用基于梯度直方图导出的模式；第三步，与第一步选中的角度模式的角度相近的角度模式。Secondary MP列表由MPM中的模式以外的一些主要角度模式构成。
由于MPM标识位的编解码顺序在MRL模式之后，ECM中MPM的编解码需依赖于MRL标识位。下表是一个示例：
Figure PCTCN2022103944-appb-000001
Figure PCTCN2022103944-appb-000002
基于模板的帧内模式推导(TIMD：Template based intra mode derivation)是一种针对亮度帧的帧内预测模式，TIMD模式由MPM列表中的候选模式和模板生成。在ECM中，如图4所示，当前块(图示为Current CU)11的左侧相邻区域和上方相邻区域构成当前块的模板(Template)区域12，而模板区域12的外侧(即左侧和上侧)设置有模板参考(reference of the template)区域13，各区域示例性的尺寸和位置如图所示。模板参考区域13可以是模板区域上方相邻的一行或左侧相邻的一列。
TIMD假定当前块和当前块的模板区域的分布特性一致，以模板参考区域的重建值作为参考行，遍历MPM和Secondary MPM中的所有模式对模板区域进行预测，得到预测结果。再计算模板区域上的重建值与每种模式的预测结果之间的误差变换绝对值和(SATD：Sum of absolute transformed differences)，选出SATD最小即最优的模式，将该模式作为当前块的TIMD模式。解码端可以通过相同的推导方式推导TIMD模式。如果序列允许使用TIMD，每个当前块需要一个标志位表示是否使用TIMD。如果为当前块选中的帧内预测模式为TIMD模式，则当前块使用TIMD模式进行预测，且剩余的亮度帧与帧内预测相关的语法元素如ISP和MPM的解码过程都可以跳过，从而大幅降低模式的编码比特。
在求得模板区域上的重建值与每种模式预测结果之间的SATD之后，可根据以下方式确定最终使用的TIMD模式：
假设mode1和mode2为MPM中用于帧内预测的两种角度模式，mode1为SATD最小的角度模式，其SATD为cost1；mode2为SATD次小的角度模式，其SATD为cost2：
当cost1×2≤cost2时，将mode1作为当前块的TIMD模式；
当cost1×2>cost2时，将对mode1和mode2的预测结果进行加权的预测模式作为当前块的TMID模式，也称TIMD融合(fusion)模式。
加权的方法和权重如下式所示：
Pred＝Pred mode1×w1+Pred mode2×w2
Figure PCTCN2022103944-appb-000003
其中，Pred为当前块使用TIMD融合模式的预测结果，Pred mode1为当前块使用mode1的预测结果，Pred mode2为当前块使用mode2的预测结果，w1和w2为根据cost1和cost2计算得到的权重。
在HEVC中，帧内预测使用距离当前块最近的上一行和左一列作为参考进行预测，如果这一行和一列的重建值与原始像素值有着较大的误差，那么当前块的预测质量也会有很大的影响。为了解决这一问题，VVC中采纳了MRL帧内预测技术，除了可以使用最近一行参考像素即参考行0(Reference line0)之外，VVC还可以使用参考行1(Reference line1)和参考行2(Reference line2)作为扩展的参考行进行帧内预测。为了减小编码复杂度，MRL仅在MPM中的非planar模式上使用。编码端对每个角度模式进行预测时，要将这三个参考行都尝试过，然后选中率失真代价(RD Cost)最小的一个参考行，选中的参考行的索引和选中的帧内预测模式的索引一样，也要编码发送到解码端。解码端解码得到参考行的索引，再根据参考行的索引选择对应的参考行进行预测。
图5所示是一个有4个参考行的例子，包括与当前块相邻的参考行0(reference line0)221即索引为0的参考行；与当前块间隔1行的参考行1(reference line1)222即索引为1的参考行；与当前块间隔2行的参考行2(reference line2)223即索引为2的参考行；以及，与当前块间隔3行的参考行3(reference line3)224即索引为3的参考行。在预测时可以只使用参考行部分的重建值。在一个示例中，将参考行0中的区域作为当前块的模板区域，则参考行0可称之为模板区域所在的参考行，参考行1至参考行3可称之为位于模板区域外侧的参考行。
本文中，参考行的索引均采用图5所示的方式编号，除了索引为0的参考行外，其他参考行是扩展参考行。
本文中，参考行虽然称之为“行”，但这是为了表述方便，一个参考行实际上包括一行和一列，预测时使用的参考行的重建值也包括一行和一列的重建值，这与业界通常的描述方法是一样的。
在ECM中，多参考行索引相关的解码方式如下表1和表2所示。其中表1用于当前块不使用TIMD的情况，表2用于当前块使用TIMD的情况。
表1
Figure PCTCN2022103944-appb-000004
Figure PCTCN2022103944-appb-000005
ECM中，MRL工具可以使用更多的参考行，为了编码当前块选中的参考行(即为当前块选中的参考行)，将多个候选的参考行的索引填入一个列表。文中，将该列表称为多参考行索引列表，简写为MRL索引列表，但也可以称为多参考行列表，候选参考行列表，参考行索引列表等。在当前块不使用TIMD的情况下，多参考行索引列表的长度为6即一共有6个位置可填入6个参考行的索引，这6个参考行的索引及其顺序是固定的，表示为MULTI_REF_LINE_IDX[6]＝{0,1,3,5,7,12}。该多参考行索引列表中，第1个位置填入的索引为0，是距离当前块最近的参考行的索引，第2个位置至第6个位置填入的索引分别为1,3,5,7,12，是距离当前块的距离从近到远的5个扩展的参考行的索引。
编码当前块选中的参考行时，因为多参考行索引列表的存在，不是对该参考行的索引直接编码，而是编码该参考行的索引在多参考行索引列表中的位置的索引，称为多参考行索引(multiRefIdx)，多参考行索引用于指示当前块选中的参考行的索引在多参考行索引列表中的位置。以MRL索引列表{0,1,3,5,7,12}为例，其中的第1个位置至第6个位置的索引分别为0至5，如果当前块选中的参考行的索引为7，索引7在多参考行索引列表中的第5个位置，则多参考行索引为4。
多参考行索引(multiRefIdx)可以使用基于上下文模型的一元截断码等方式编码，但不局限于此。多参考行索引的值越小，码长越少，解码越快。
MRL模式也可以与TIMD模式同时使用，在使用TIMD的情况下，MRL索引列表长度为3，可填入3个参考行的索引且索引之间的顺序固定，表示为MULTI_REF_LINE_IDX[3]＝{0,1,3}，其中，0是模板区域的参考行的索引，1,3为扩展的参考行的索引。
多参考行索引的编解码方式如表2所示：
表2
Figure PCTCN2022103944-appb-000006
Figure PCTCN2022103944-appb-000007
为了方便理解，上文将MRL在使用TIMD和不使用TIMD的情况下分为了两张表进行举例，上述两张表也可以合为一个表。
在不同的标准中，利用当前块周围的块，导出一个最可能预测模式的列表(MPM)的技术可能具有不同的名称，例如，在AV2(AVM)中称为自适应帧内模式编码(AIMC：Adaptive Intra Mode Coding),在AVS3中，在屏幕内容编码情况下，称为基于频数信息的帧内编码(FIMC：Frequency-based Intra Mode Coding)。在非屏幕内容编码情况下，类似MPM的这个技术常开。而利用多参考行进行帧内预测的多参考行技术(MRL)，在AV2(AVM)中称为用于帧内预测的多参考前选择(MRLS：Multiple reference line selection for intra prediction)。这仅仅是名称的不同，本实施例使用MPM、MRL时，同样覆盖到其他标准中的这些实质相同的技术。
与VVC相比，ECM拓展了MRL的参考行候选个数，但在帧内帧编码上仅仅带来0.09％的编码性能提升。经研究，这是由于MRL候选索引行个数的增加，编码MRL模式需要更多的标识位置，带来的额外负担也增加。而从上述表1和表2所示的MRL解码方式可以看出，多参考行索引列表由指定的固定顺序的索引构成，选中的参考行的索引在该表越靠前的位置，编码多参考行索引代价也越小。
本公开一实施例提供了一种多参考行索引列表排序方法，该方法即可以用于编码端，也可以用于解码端。如图6所示，所述方法包括：
步骤110，根据当前块位于模板区域外侧的至少部分参考行对所述模板区域分别进行预测，并计算所述模板区域的重建值和预测得到的预测值之间的误差；
上述误差可以用绝对误差和SAD表示，或用误差变换绝对值和SATD表示，但不局限于此，也可以用差值的平方和(SSD)、平均绝对差值(MAD)、平均平方误差(MSE)等表示。
步骤120，按照所述误差从小到大的顺序，将所述误差对应的参考行的索引填入当前块的MRL索引列表，得到基于当前块排序后的MRL索引列表。
之前的实施例中，MRL索引列表是编解码系统设定的，MRL索引列表中填入的参考行的索引以及顺序是固定的。本公开实施例使用基于模板的方式，根据当前块位于模板区域外侧的至少部分参考行对模板区域分别进行预测，并计算模板区域的重建值和预测得到的预测值之间的误差，照误差从小到大的顺序将所述误差对应的参考行的索引填入当前块的MRL索引列表，得到基于当前块排序后的MRL索引列表。因此按照本公开实施例构建的是一个块级的MRL索引列表，其中参考行的索引以及顺序是基于当前块模板区域的分布特征确定的，因而当前块选中的参考行更可能出现在MRL索引列表中，也更可能出现在MRL索引列表中靠前的位置，即需要编码的多参考行索引的值更可能是相对较小的值，使得码长少，占用带宽少，也使得解码速度变快，从而提高了编码性能。
本公开一示例性的实施例中，所述根据当前块位于模板区域外的至少部分参考行分别对模板区域进行预测，包括：对所述至少部分参考行中的每一参考行，使用该参考行的重建值和当前块选中的帧内预测模式对所述模板区域进行预测。本实施例方法用于编码端时，可以在帧内预测模式选择完成，为当前块选中了一种帧内预测模式(此处指平均模式、平面模式和角度模式等传统的帧内预测模式)后再基于当前块选中的帧内预测模式，结合每一个参与排序的参考行对模板区域进行预测。本实施例方法用于解码端时，可以在解码当前块的帧内预测相关语法元素，确定了当前块选中的帧内预测模式之后，再进行上述预测，得到基于当前块排序后的MRL索引列表后，再根据解码得到的多参考行索引就可以确定当前块选中的参考行。
本公开一示例性的实施例中，所述将所述误差对应的参考行的索引填入当前块的MRL索引列表，是从所述MRL索引列表的第K 1个位置开始将所述误差对应的参考行的索引逐位置填入，最多可填入到第K 2个位置，其中，2≤K 1≤K 2≤M，M为所述MRL索引列表的长度。实际填入MRL索引列表的索引可以小于等于M。
在当前块的MRL索引列表中，可以填入模板区域所在的参考行，如图5所示的参考行21，其索引为0；也可以填入模板区域外侧的参考行，如图5所示的参考行22、23和24，这三个参考行的索引分别为1，2，3。对当前块的MRL索引列表排序时，并不使用模板区域所在的参考行进行预测，因而 模板区域所在的参考行的索引不参与排序，而这些参考行被选中的可能也较大，本实施例将其直接填入在MRL索引列表靠前的一个或多个位置，从第K 1个位置开始将参与排序的参考行的索引逐位置填入，K 1如可以为2，3但不局限于此。
本公开一示例性的实施例中，所述方法还包括：所述基于当前块排序后的MRL索引列表中，从第1个位置到第K 1-1个不参与排序的位置，填入的是当前块的模板区域所在的参考行的索引，其中，第1个位置填入的参考行的索引为0。在本实施例中，模板区域只限于距离当前块最近的一行和一列时，只需要填入距离当前块最近的一个参考行的索引0。但在其他实施例中，模板区域也可以扩展，如扩展到当前块上方的第二行和左侧的第二列，或者更大的区域，也即设置在距离当前块最近的多个参考行上。此时，可以将更多的参考行的索引填入到MRL索引列表中，但是，并不是与模板区域重叠的所有参考行的索引都要填入MRL索引列表，也可以选择其中的一部分填入MRL索引列表。
本公开一示例性的实施例中，所述将所述误差对应的参考行的索引填入当前块的MRL索引列表，是从所述MRL索引列表的第1个位置开始将所述误差对应的参考行的索引逐位置填入，最多可填入到第K 2个位置，其中，1≤K 2≤M，M为所述MRL索引列表的长度。与前述实施例不同的是，本实施例并不需要在MRL索引列表中填入当前块的模板区域所在的参考行的索引，例如，假定模板区域所在的参考行只有一行，当前块选中的参考行的索引为0即不使用扩展的参考行时，该选中的参考行的索引可以不需要使用多参考行索引来表示，将MRL使用标志(flag)置为假就可以表示。因此MRL索引列表可以不用填入索引0，而参与排序的参考行的索引可以按照误差从小到大的顺序从MRL索引列表的第1个位置开始填入。
本公开一示例性的实施例中，在K 2<M的情况下，从所述MRL索引列表的第K 2+1个位置开始到第M个位置，按照距离当前块从近到远的顺序逐位置填入M-K 2个参考行的索引，所述M-K 2个参考行位于所述模板区域外侧，由系统指定，且索引不同于已填入所述MRL索引列表的索引。
本实施例是在MRL索引列表靠后的一个或多个位置上，不再按照误差从小到大的顺序将所述误差对应的参考行的索引填入，而是填入模板区域外侧的指定的参考行。例如，MRL索引列表的长度M为6，根据误差排序的参考行的索引只填入其中的第2个位置和第3个位置。在一个示例中，设定的MRL索引列表为{0,1,3,5,7,12}，1,3,5,7,12为指定的参考行的索引。本实施例根据索引为1～12的参考行进行预测，假定误差最小的两行是索引为5和7的参考行，因为这2个参考行原来就是指定的参考行，因而基于当前块排序后的MRL索引列表为{0,5,7,1,3,12}，即将设定的MRL索引列表中指定的参考行的索引1,3,12填入到排序后的MRL索引列表靠后的位置。如果误差最小的两行是索引为4和6的参考行，由于这两行不是指定的参考行，故排序后的MRL索引列表为{0,4,6,1,3,5}，即指定的参考行的索引是按照参考行距离当前块从近到远的顺序逐位置填入的。
本公开一示例性的实施例中，所述当前块的模板区域设置在距离当前块最近的一个参考行上，即设置在索引为0的参考行：或者，所述当前块的模板区域设置在距离当前块最近的多个参考行上；如设置在索引为0,1的两个参考行上，或者设置在索引为0,1,2的三个参考行上，或者设置在索引为0,1,2,3的四个参考行上，等等。
本公开一示例性的实施例中，所述当前块位于模板区域外侧的至少部分参考行包括：
1)所述模板区域外侧距离当前块最近的N’个参考行；或者
2)所述模板区域外侧距离当前块最近的N’个参考行中的奇数行；或者
3)所述模板区域外侧距离当前块最近的N’个参考行中的偶数行；或者
4)所述模板区域外侧距离当前块最近的N’个参考行中，通过邻近行之间的误差比较选出的N个参考行，N≤N’；或者
5)所述模板区域外侧距离当前块最近的N’个参考行中指定的N个参考行；
其中，N’≤N max，N max是允许参与排序的参考行的最大个数。
在排序时，首先需要确定参考排序的参考行的数量，以及具体选择哪些参考行。本实施例给出了多种选择方式。
本实施例中，假定当前块使用TIMD的情况下设定的N max和当前块不使用TIMD的情况下设定的N max均为12，模板区域处于距离当前块最近的参考行即索引为0的参考行中，且不允许当前块的参考 行超出所在CTU的上边界，则上述第1种方式是选择将模板区域外侧距离当前块最近的N’个参考行参与排序，如果模板区域外侧到CTU的上边界之间的参考行的个数大于或等于12个行，则N’＝12，即选出模板区域外侧距离当前块最近的12个参考行参与排序。如果没有12个的参考行则N’等于模板区域外侧到CTU的上边界之间的参考行的个数。第2种和第3种方式分别是从第一种方式确定的参考行中再选出其中的奇数行和偶数行，以简化运算。
在本实施例的一个示例中，本实施例的第四种方式即从模板区域外侧距离当前块最近的N’个参考行中，通过邻近行之间的误差比较选出N个参考行，可以采用以下方式来实现：将所述模板区域外侧距离当前块最近的一个参考行作为选出的第一个参考行，将新选出的该参考行与其邻近参考行逐一比较，将距离该个参考行最近且两个参考行之间误差大于设定阈值的一邻近参考行作为新选出的一个参考行，如此循环，直至已对N’个参考行比较和选择。其中，所述误差指两个参考行(参与比较的两个参考行)的重建值之间的误差，或者是指以两个参考行的重建值为参考，对模板区域分别进行预测得到的预测值之间的误差。
以下用一个示例说明比较的过程，为了表述方便，将模板区域外侧的参考行按照到当前块的距离从近到远的顺序编号，距离当前块最近的参考行称为模板区域外侧的第1个参考行，简称为第1个参考行；将距离当前块次近的参考行称为模板区域外侧的第2个参考行，简称为第2个参考行；依此类推。再假定模板区域到CTU上边界之间的参考行数量大于12。
该比较过程如下：将模板区域外侧第1个参考行选出后，比较第1个参考行与其相邻行即第2个参考行的误差，如果误差小于设定阈值，则继续比较第1个参考行与第3个参考行的误差，如果误差大于设定阈值，则将该第3个参考行作为新选出的一行；然后比较该第3个参考行与其相邻行(即第4个参考行)的误差，如果误差大于设定阈值，将该第4个参考行作为新选出的一行；再比较该第4个参考行与其相邻行(即第5个参考行)的误差，依此类推，只到比较完第12个参考行，如果该第12个参考行与最近选出的参考行之间的误差大于设定阈值，则将第12个参考行选出参与排序，否则不选择第12个参考行。
可以看出，采用以上4种方式时，本公开实施例的MRL索引列表排序方法，并不仅限于排序本身，还可以选择参与排序的参考行，进而也选择了索引填入MRL索引列表的参考行。这有利于找到最优的参考行，提升视频编码的效率。
但本公开也可以使用指定的参考行进行排序。上述第5种方式中，参与排序的参考行是从模板区域外侧距离当前块最近的N’个参考行中指定的N个参考行，例如指定的索引为1,3,5,7,12的参考行。此时相当于对设定的MRL索引列表中的参考行进行排序，并没有引入新的参考行。
在本实施例的一个示例中，所述模板区域外侧距离当前块最近的N’个参考行是不超过当前块所在编码树单元CTU上边界的参考行；
在本实施例的一个示例中，当前块使用基于模板的帧内模式推导TIMD的情况下设定的N max和当前块不使用TIMD的情况下设定的N max均大于或等于3。在一个示例中，当前块使用TIMD的情况下设定的N max为12，而当前块不使用TIMD的情况下N max也可以设为12。对于N max的具体取值本公开不做局限，如可以是5，6，7，8，9，10，11，12，13，14，15以及大于15的值，如18，24，36等。
本公开实施例还提供了一种视频解码方法，如图7所示，包括：
步骤210，解码码流中当前块的帧内预测相关语法元素，在当前块允许使用多参考行MRL的情况下，继续解码当前块的多参考行索引；
如前所述，多参考行索引用于指示当前块选中的参考行的索引在MRL索引列表中的位置。
步骤220，根据解码结果确定允许对当前块的MRL索引列表排序的情况下，按照本公开任一实施例所述的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；
步骤230，根据所述多参考行索引和基于当前块排序后的MRL索引列表，确定当前块选中的参考行，根据当前块选中的参考行对当前块进行帧内预测。
本实施例在解码过程中，在确定允许对当前块的MRL索引列表排序的情况下，先根据本公开上述实施例的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；再根据解码得到的多参考行索引和基于当前块排序后的MRL索引列表，确定当前块选中的参考行，进而根据 当前块选中的参考行对当前块进行帧内预测。本实施例可以使用基于当前块排序的MRL索引列表进行多参考行索引的解码，使得当前块选中的参考行有更大可能位于MRL索引列表靠前的位置，从而可以节约编码开销，提升编码性能。
在本公开一示例性的实施例中，解码码流中当前块的帧内预测相关语法元素时解码当前块选中的帧内预测模式，以用于对当前块的MRL索引列表排序；根据当前块选中的参考行对当前块进行帧内预测时，根据当前块选中的参考行和当前块选中的帧内预测模式对当前块进行帧内预测。
在本公开一示例性的实施例中，所述方法还包括：根据解码结果确定不允许对当前块的MRL索引列表排序的情况下，根据所述多参考行索引和设定的MRL索引列表确定当前块选中的参考行，根据当前块选中的参考行对当前块进行帧内预测。本实施例在确定不允许对当前块的MRL索引列表排序的情况下，使用设定的MRL索引列表来确定多参考行索引所指向的参考行。
本实施例的一个示例中，确定当前块不使用TIMD的情况下，所述设定的MRL索引列表为{0,1,3,5,7,12}；确定当前块使用TIMD的情况下，所述设定的MRL索引列表为{0,1,3}或{0,1,2}。
对于是否允许对当前块的MRL索引列表排序，可以根据解码多参考行索引确定，也可以直接使用一个标志来表示。
在本公开一示例性的实施例中，所述方法还包括：确定允许对当前块的MRL索引列表排序的情况下，将当前块表示允许对MRL索引列表排序的标志置为真。
在本公开另一示例性的实施例中，将以下条件中的任意一种或更多种作为不允许对当前块的MRL索引列表排序的充分条件：
当前块不使用多参考行MRL；
多参考行索引指示的是MRL索引列表中不参与排序的位置，如第1个位置；
当前帧非亮度帧；
当前块位于编码树单元CTU的上边界；
当前块选中的帧内预测模式为指定模式，所述指定模式包括以下模式中的任意一种或更多种：平面模式、有一个模式为平面模式的TIMD融合模式、平均模式、水平角度模式、垂直角度模式、宽角度模式；
当前块的大小不满足设定条件；
当前块的长宽比不满足设定条件。
在不允许对当前块的MRL索引列表排序的条件(不限于以上列出的条件)均不成立的情况下，确定允许对当前块的MRL索引列表排序。
在一个示例中，还可以把当前块使用TIMD和当前块不使用TIMD中的一种，作为不允许对当前块的MRL索引列表排序的充分条件。也即此时只允许在当前块使用TIMD或当前块不使用TIMD的情况下，对MRL索引列表排序。
在本公开一示例性的实施例中，在当前块使用MRL的情况下，所述方法还包括：解码当前块能够与MRL同时使用的部分变换模式的语法元素，跳过不能与MRL同时使用的部分变换模式的语法元素。
本实施例在当前块使用MRL的情况下可以跳过一些变换模式。MRL模式在所有帧内预测模式中处于较为靠后的位置，例如，需要确定当前模式不是DIMD,BDPCM,MIP,TIMD后才会判断是否为MRL。在VVC中，残差信号可以使用一次变换和二次变换使得变换后的残差更易于编码。其中一次变换包括DCT2变换和DST7。而二次变换是在一次变换使用DCT2的基础上对一次变换后的残差进一步变换。在ECM中,一次变换的模式也进行了进一步拓展。考虑到当选中不同模式是需要额外比特来表示会带来码字的开销，同时在编码端遍历这些变换模式也会增加编码时间，所以当帧内预测选中了多参考行模式时，可以限制其一次变换只使用DCT2模式且可以使用二次变换，或是只可以使用一次变换，或是只可以使用DCT2变换，这样当在编码对应的变换模式的索引时，可以根据是否已经选中了MRL模式，对变换模式的编码进行跳过。
在本公开一示例性的实施例中，在当前块使用MRL的情况下，所述方法还包括：在解码当前块 选中的帧内预测模式的过程中，允许解码与第二MPM相关的语法元素。本实施例允许MRL和第二MPM模式同时使用。
在本公开一示例性的实施例中，所述解码当前块的多参考行索引，包括：在当前块使用TIMD的情况下和不使用TIMD的情况下，根据不同的MRL索引列表解码当前块的多参考行索引。
本公开一实施例还提供了一种视频编码方法，如图8所示，包括：
步骤310，对当前块进行帧内预测，确定当前块选中的参考行；
步骤320，确定允许对当前块的MRL索引列表排序的情况下，按照如权利要求1至11中任一所述的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；
步骤330，根据当前块选中的参考行的索引在基于当前块排序后的MRL索引列表中的位置，确定当前块的多参考行索引并编码。
本实施例在编码过程中，在确定允许对当前块的MRL索引列表排序的情况下，先根据本公开上述实施例的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；再根据当前块选中的参考行的索引在基于当前块排序后的MRL索引列表中的位置，确定当前块的多参考行索引并编码。本实施例可以使用基于当前块排序的MRL索引列表进行多参考行索引的编码，使得当前块选中的参考行有更大可能位于MRL索引列表靠前的位置，从而节约多参考行索引的编码开销，提升编码性能。
在本公开一示例性的实施例中，对当前块的MRL索引列表排序之前，所述方法还包括：确定当前块选中的帧内预测模式。
在本公开一示例性的实施例中，所述方法还包括：确定不允许对当前块的MRL索引列表排序的情况下，根据当前块选中的参考行的索引在设定的MRL索引列表中的位置，确定当前块的多参考行索引并编码。本实施例在确定不允许对当前块的MRL索引列表排序的情况下，使用选中的参考行在设定的MRL索引列表中的位置确定当前块的多参考行索引。
在本公开一示例性的实施例中，所述方法还包括：确定允许对当前块的MRL索引列表排序的情况下，将当前块表示允许对MRL索引列表排序的标志置为真。本实施例对于是否允许对当前块的MRL索引列表排序，是直接使用一个标志来表示。
在本公开一示例性的实施例中，将以下条件中的任意一种或更多种，作为不允许对当前块的MRL索引列表排序的充分条件：
当前块不使用MRL；
当前块选中的参考行位于当前块的模板区域；
当前帧非亮度帧；
当前块位于编码树单元CTU的上边界；
当前块选中的帧内预测模式为指定模式，所述指定模式包括以下模式中的任意一种或更多种：平面模式、有一个模式为平面模式的TIMD融合模式、平均模式、水平角度模式、垂直角度模式、宽角度模式；
当前块的大小不满足设定条件；
当前块的长宽比不满足设定条件；
当前块使用TIMD和当前块不使用TIMD中的一种。
在不允许对当前块的MRL索引列表排序的条件(不限于以上几种)均不成立的情况下，确定允许对当前块的MRL索引列表排序。
在本公开一示例性的实施例中，在当前块使用TIMD的情况下和不使用TIMD的情况下，使用不同的MRL索引列表。
所述确定当前块的多参考行索引并编码，包括：
在当前块使用TIMD的情况下，根据当前块选中的参考行的索引在第一MRL索引列表中的位置，确定当前块的多参考行索引并编码；
在当前块不使用TIMD的情况下，根据当前块选中的参考行的索引在第二MRL索引列表中的位置，确定当前块的多参考行索引并编码；
其中，所述第二MRL索引列表的长度大于或等于所述第一MRL索引列表的长度。
在本公开一示例性的实施例中，在当前块使用MRL的情况下，所述方法还包括：编码当前块能够与MRL同时使用的部分变换模式的语法元素，跳过不能与MRL同时使用的部分变换模式的编码。
在本公开一示例性的实施例中，在当前块使用MRL，且当前块选中的模式非TIMD模式的情况下。所述方法还包括：编码所述多参考行索引之后，继续编码当前块选中的帧内预测模式，且在编码当前块选中的帧内预测模式的过程中，允许编码与第二MPM相关的语法元素。
本公开一实施例还提供了一种码流，其中，所述码流中包括当前块的多参考行索引，所述多参考行索引用于指示当前块选中的参考行的索引在MRL索引列表中的位置，所述MRL索引列表是基于当前块排序后的MRL索引列表，或者是设定的MRL索引列表。本实施例的码流可以通过如本公开任一实施例所述的视频编码方法生成。
本公开一实施例还提供了一种多参考行索引列表排序装置，如图9所示，包括处理器71以及存储有计算机程序的存储器73，其中，所述处理器71执行所述计算机程序时能够实现如本公开任一实施例所述的多参考行索引列表排序方法。
本公开一实施例还提供了一种视频解码装置，参见图9，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的视频解码方法。
本公开一实施例还提供了一种视频编码装置，参见图9，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的视频编码方法。
本公开上述实施例的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)、微处理器等等，也可以是其他常规的处理器等；所述处理器还可以是数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)、离散逻辑或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件；也可以是上述器件的组合。即上述实施例的处理器可以是实现本发明实施例中公开的各方法、步骤及逻辑框图的任何处理器件或器件组合。如果部分地以软件来实施本公开实施例，那么可将用于软件的指令存储在合适的非易失性计算机可读存储媒体中，且可使用一个或多个处理器在硬件中执行所述指令从而实施本公开实施例的方法。
本公开一实施例还提供了一种视频编解码系统，其中，包括本公开任一实施例所述的视频编码装置和本公开任一实施例所述的视频解码装置。
本公开一实施例还提供了一种非瞬态计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其中，所述计算机程序时被处理器执行时实现如本公开任一实施例所述的多参考行索引列表排序方法，或实现本公开任一实施例所述的视频解码方法，或实现本公开任一实施例所述的视频编码方法。
本公开一实施例提供了一种视频解码方法，在解码过程中对当前块的多参考行索引列表排序。本实施例基于模板区域的排序是编码器和解码器都需要进行的操作，它作用于从MRL索引列表中，通过多参考行索引表示选中的参考行在MRL索引列表中的位置，结合基于当前块排序的MRL索引列表共同确定当前块选中的参考行，再用确定的该参考行和选中的帧内预测模式(传统预测模式)进行预测。
本实施例用一个具体的示例阐述如何使用模板区域上的预测值(pred)与重建值(reco)的绝对误差和(SAD：the sum of absolute difference)对MRL索引列表进行排序。(x,-1),(-1,y)分别是相对当前块左上角(0,0)位置的坐标，此实施例中，当前块的1行1列的模板区域，请参见图10所示的带有剖面线的区域。该图中示出了当前块模板区域外侧的6个的参考行。
本实施例方法包括：
第一步
解码器解析帧内预测模式，确认当解码器解析出的MRL相关语法元素，在当前块上使用时或不使用MRL，以及解码出当前块的多参考行索引。
本实施例相关的语法表如下表所示：
Figure PCTCN2022103944-appb-000008
Figure PCTCN2022103944-appb-000009
表中，intra_luma_mrl_tab_idx定义了当前块不使用TIMD的情况下(即！intra_timd_flag)的多参考行索引，以指示当前块选中的参考行的索引在排序后的MRL索引列表中的位置，它的取值范围为0到5。intra_timd_luma_ref_idx则定义了当前块使用TIMD的情况下的多参考行索引，以指示当前块选中的参考行的索引在设定的MRL索引列表中的位置，它的取值范围为0到2。这两种情况下的多参考行索引均可采用基于上下文模型的截断一元码编解码。
本示例是在当前块使用TIMD的情况下，不对设定的MRL索引列表排序。但在其他示例中，在当前块使用TIMD的情况下，也可以生成对当前块的MRL索引列表排序。在当前块使用TIMD的情况下对MRL索引列表排序时，可以只对索引填入设定的MRL索引列表中的参考行排序，也可以引入更多的参考行，例如和当前块不使用TIMD的情况下相同的参考行参与排序。此时，可以使用一个类似于intra_luma_mrl_tab_idx的语法元素将用于替代intra_timd_luma_ref_idx。
本文表中的timd与文字中的TIMD是同一概念，其他概念类似。
第二步
解码阶段结束后，在对当前块进行预测前，如果当前块的多参考行索引不是0，即选中的参考行的索引不为0，则不使用设定的MRL索引列表，而是需要对当前块的MRL索引列表排序。排序只针对扩展的参考行(本实施例即索引不为0的参考行)进行。
排序的过程如下。
首先，确定当前块在图像中所处位置最多可以允许多少个扩展参考行，原则上，当前块可以使用的上方参考行，不应超出上方的CTU边界。在ECM中，CTU最大可以为256x256。在当前块左上角所处于图像中位置的纵坐标y对256取余数时，得到的余数则为上方在CTU边界内的参考行的数量。同时，设定MRL在非TIMD模式下最多可以使用13个参考行(reference line 0～12)，则可参与排序的参考行为reference line 1～N，即索引为1～N的参考行。其中N＝min(12,(x％256)-1))。在其他实施例中，如果硬件支持，使用的参考行也可以超过CTU边界。
reference line 1～N这N个参考行在reference line 0所在行的模板区域外侧，对其中每一个参考行，使用选中的帧内预测模式对模板区域进行预测。由于reference line 0的模板区域已完成重建，此时可根据重建值与预测得到的预测值求出SAD。将N个SAD值按照升序排序，并将对应计算出的SAD最小的M个参考行的索引填入MRL索引列表。
具体地，MRL索引列表在当前块不使用TIMD的情况下长度为6，在使用TIMD的情况下长度为3，而这两个列表设定的填充值分别为{0,1,3,5,7,12}和{0,1,3}，分别代表索引为0,1,3,5,7,12的参考行和索引为0,1,3的参考行。本实施例的排序是对reference line 1～N进行排序，因而MRL索引列表中第1个位置填入的索引0始终不变，只会将用排序后的扩展的参考行填入后面的位置，在当前块不使用TIMD的情况下，最多可填入5个索引；在当前块使用TIMD的情况下，最多可填入2个索引。
在排序的过程中，应考虑到在有的情况下，相邻的两行待排序的参考行可能有着相似的重建值，这时使用这两行预测出的结果近似，此时可以跳过其中一行从而去除冗余。这里可以有两种跳过方法，
方法一.计算邻近两个参考行对应位置上重建值之间的SAD值，当SAD值小于一个预设阈值时，认为这两个参考行相似度过大应跳过其中的一个参考行。如果邻近的该参考行被跳过，那么将已选出的该参考行继续与下一个最近的参考行进行阈值比较。
方法二.分别使用两个参考行对模板区域进行预测，计算模板区域上的预测结果之间的SAD值，当SAD值小于一个预设阈值时，认为这两行相似度过大应跳过其中一行。
上述跳过方法可以跳过这两行中任意一行，但考虑到通常情况下距离当前块更近的参考行更适合作为参考行，本实施例是跳过较远的参考行。
由于ECM中，MRL和TIMD与MPM技术可以同时使用，在TIMD下，有时决策出来的帧内预测模式是TIMD融合模式，而MRL与Planar模式共同作用通常不会带来额外性能提升，所以当MRL选中时，若遇到帧内预测模式为Planar或TIMD融合(fusion)模式下有一个模式是Planar，则可以不排序。除Planar模式，也可以在一些特定的角度或模式下不排序，例如DC模式下不排序，宽角度下不排序,水平和垂直角度模式下不排序等等。宽角度模式可以等价成在长方形预测块时不排序。
不排序时，直接使用设定初始的参考行列表如{1,3,5,7,12}而不对这些列表中的参考行进行基于SAD的重排序。
本实施例可以跳过一些变换模式，当前的MRL编码在所有帧内模式中处于较为靠后的位置，在ECM中，需要确定当前模式不是DIMD,BDPCM,MIP,TIMD后才会判断是否为MRL。而考虑到后续变换模式编码还需要用更多的标识位标识一次变换和二次变换，为了减小这一方面的编码开销，也可以限制MRL只与部分变换模式组合使用，例如只可以与DCT2的一次变换一起使用或只可以与DCT2的一次变换以及二次变换一起使用。例如，限制一次变换下MRL只能与DCT2共同使用时，解码变换模式的语法元素应查看MRL是否已经选中，若选中则不解码一次变换模式。
相应的语法如下：
Figure PCTCN2022103944-appb-000010
Figure PCTCN2022103944-appb-000011
上表中填写有“intra_luma_ref_tab_idx！＝0&&”的一栏中，定义了一些条件以判断是否解码mts_idx(一次变换选中的变换模式索引)，其中包括多参考行索引不为0。
在另一示例中，限制一次变换下MRL不与二次变换共同使用时，解码变换模式的语法元素应查看MRL是否已经选中，若选中则不解码二次变换模式。相应语法如下：
Figure PCTCN2022103944-appb-000012
Figure PCTCN2022103944-appb-000013
上表中填写有“intra_luma_ref_tab_idx！＝0&&”的一栏中，定义了一些条件判断是否解码lfnst_idx(二次变换选中的变换模式索引)，其中包括多参考行索引不为0
第三步
根据重排序的多参考行索引列表和解码得到的列表索引确定参考行并对当前块进行预测。
本实施例在帧内预测使用多参考行索引列表排序时，通过使用模板区域的预测值与重建值之间的误差，按照升序排序，得到基于当前块排序的多参考行索引列表，从而使得更有可能被选中用来预测的参考行排在多参考行索引列表的更靠前的位置，节约编码开销。本实施例在排序的过程中，通过阈值设置来跳过一些可能重复的待排序的参考行，可以减少冗余并提高排序效率。本实施例模板区域所在的参考行以及小于该参考行的参考行不能加入排序，可以保持其在候选列表中的原有位置。
在本实施例的基础上，可以增加可排序的参考行。本实施例虽然使用最多12个扩展的参考行参与排序。实际上，在不考虑复杂度的情况下，可以取消这一限制使得可以用来排序的超过12行以外的参考行，以此来获取更好的编解码性能。
此外，可以增加模板区域的大小，模板区域不局限于距离当前块最近的参考行，而是可以继续扩大，例如扩展到上方模板区域的右侧、左侧模板区域的下方以及左上角相交的灰色部分扩展。也可以向上方模板区域的上方，左侧模板区域的左侧继续拓展，这样会使用到更多参考行的区域，例如进一步用到的reference line 1的区域作为模板区域时，reference line 1本身不能参与排序。
在ECM-4.0参考软件上，使用本实施例所述的方法，对当前块不使用TIMD情况下的MRL索引列表进行重排序，可以在参考软件上进一步带来如下的性能提升，
Figure PCTCN2022103944-appb-000014
使用本实施例上述方法，对当前块使用TIMD的情况下和不使用TIMD的情况下的MRL索引列表都进行排序，可以在参考软件上进一步带来如下的性能提升，
Figure PCTCN2022103944-appb-000015
以上测试在当前块使用TIMD的情况下也使用reference line 1～12参与排序，只保留其中SAD最小的两个参考行line_a,line_b填入排序后的MRL索引列表:{0,line_a,line_b}。
表中的参数如下：
EncT:Encoding Time，编码时间,10X％代表当集成了参考行排序技术后，与没集成前相比，编码时间为10X％,这意味有X％的编码时间增加。
DecT:Decoding Time，解码时间，10X％代表当集成了参考行排序技术后，与没集成前相比，解码时间为10X％,这意味有X％的解码时间增加。
ClassA1和Class A2是分辨率为3840x2160的测试视频序列，ClassB为1920x1080分辨率的测试序列，ClassC为832x480，ClassD为416x240,ClassE为1280x720；ClassF为若干个不同分辨率的屏幕内容序列(Screen content)。
All intra表示全帧内帧配置的测试配置。
在以上一个或多个示例性实施例中，所描述的功能可以硬件、软件、固件或其任一组合来实施。如果以软件实施，那么功能可作为一个或多个指令或代码存储在计算机可读介质上或经由计算机可读介质传输，且由基于硬件的处理单元执行。计算机可读介质可包含对应于例如数据存储介质等有形介质的计算机可读存储介质，或包含促进计算机程序例如根据通信协议从一处传送到另一处的任何介质的通信介质。以此方式，计算机可读介质通常可对应于非暂时性的有形计算机可读存储介质或例如信号或载波等通信介质。数据存储介质可为可由一个或多个计算机或者一个或多个处理器存取以检索用于实施本公开中描述的技术的指令、代码和/或数据结构的任何可用介质。计算机程序产品可包含计算机可读介质。
举例来说且并非限制，此类计算机可读存储介质可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来以指令或数据结构的形式存储所要程序代码且可由计算机存取的任何其它介质。而且，还可以将任何连接称作计算机可读介质举例来说，如果使用同轴电缆、光纤电缆、双绞线、数字订户线(DSL)或例如红外线、无线电及微波等无线技术从网站、服务器或其它远程源传输指令，则同轴电缆、光纤电缆、双纹线、DSL或例如红外线、无线电及微波等无线技术包含于介质的定义中。然而应了解，计算机可读存储介质和数据存储介质不包含连接、载波、信号或其它瞬时(瞬态)介质，而是针对非瞬时有形存储介质。如本文中所使用，磁盘及光盘包含压缩光盘(CD)、激光光盘、光学光盘、数字多功能光盘(DVD)、软磁盘或蓝光光盘等，其中磁盘通常以磁性方式再生数据，而光盘使用激光以光学方式再生数据。上文的组合也应包含在计算机可读介质的范围内。
可由例如一个或多个数字信号理器(DSP)、通用微处理器、专用集成电路(ASIC)现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一个或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指上述结构或适合于实施本文中所描述的技术的任一其它结构中的任一者。另外， 在一些方面中，本文描述的功能性可提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或并入在组合式编解码器中。并且，可将所述技术完全实施于一个或多个电路或逻辑元件中。
本公开实施例的技术方案可在广泛多种装置或设备中实施，包含无线手机、集成电路(IC)或一组IC(例如，芯片组)。本公开实施例中描各种组件、模块或单元以强调经配置以执行所描述的技术的装置的功能方面，但不一定需要通过不同硬件单元来实现。而是，如上所述，各种单元可在编解码器硬件单元中组合或由互操作硬件单元(包含如上所述的一个或多个处理器)的集合结合合适软件和/或固件来提供。

Claims (34)

  1. 一种多参考行索引列表排序方法，包括：
    根据当前块位于模板区域外侧的至少部分参考行对所述模板区域分别进行预测，计算所述模板区域的重建值和预测得到的预测值之间的误差；
    按照所述误差从小到大的顺序，将所述误差对应的参考行的索引填入当前块的多参考行MRL索引列表，得到基于当前块排序后的MRL索引列表。
  2. 如权利要求1所述的方法，其中：
    所述根据当前块位于模板区域外的至少部分参考行分别对模板区域进行预测，包括：对所述至少部分参考行中的每一参考行，使用该参考行的重建值和当前块选中的帧内预测模式对所述模板区域进行预测。
  3. 如权利要求1所述的方法，其中：
    所述将所述误差对应的参考行的索引填入当前块的MRL索引列表，是从所述MRL索引列表的第K 1个位置开始将所述误差对应的参考行的索引逐位置填入，最多可填入到第K 2个位置，其中，2≤K 1≤K 2≤M，M为所述MRL索引列表的长度。
  4. 如权利要求3所述的方法，其中：
    所述方法还包括：在所述基于当前块排序后的MRL索引列表中，从第1个位置到第K 1-1个不参与排序的位置，填入当前块的模板区域所在的参考行的索引，其中，第1个位置填入的参考行的索引为0。
  5. 如权利要求1所述的方法，其中：
    所述将所述误差对应的参考行的索引填入当前块的MRL索引列表，是从所述MRL索引列表的第1个位置开始将所述误差对应的参考行的索引逐位置填入，最多可填入到第K 2个位置，其中，1≤K 2≤M，M为所述MRL索引列表的长度。
  6. 如权利要求3或5所述的方法，其中：
    所述方法还包括：在K 2<M的情况下，从所述MRL索引列表的第K 2+1个位置开始到第M个位置，按照距离当前块从近到远的顺序逐位置填入M-K 2个参考行的索引，所述M-K 2个参考行位于所述模板区域外侧、由系统指定，且索引不同于已填入所述MRL索引列表的索引。
  7. 如权利要求1所述的方法，其中：
    所述当前块位于模板区域外侧的至少部分参考行包括：
    所述模板区域外侧距离当前块最近的N’个参考行；或者
    所述模板区域外侧距离当前块最近的N’个参考行中的奇数行；或者
    所述模板区域外侧距离当前块最近的N’个参考行中的偶数行；或者
    所述模板区域外侧距离当前块最近的N’个参考行中，通过邻近行之间的误差比较选出的N个参考行，N≤N’；或者
    所述模板区域外侧距离当前块最近的N’个参考行中指定的N个参考行；
    其中，N’≤N max，N max是允许参与排序的参考行的最大个数。
  8. 如权利要求7所述的方法，其中：
    所述模板区域外侧距离当前块最近的N’个参考行是不超过当前块所在编码树单元CTU上边界的参考行；
    当前块使用基于模板的帧内模式推导TIMD的情况下设定的N max和当前块不使用TIMD的情况下设定的N max均大于或等于3。
  9. 如权利要求7所述的方法，其中：
    所述通过邻近参考行之间的误差比较选出N个参考行，采用的方式如下：
    将所述模板区域外侧距离当前块最近的一个参考行作为选出的第一个参考行，将新选出的该参考行与其邻近的参考行逐一比较，将距离该参考行最近且两个参考行之间误差大于设定阈值的一邻近参考行作为新选出的一个参考行，如此循环，直至已对N’个参考行比较和选择；其中，所述误差指两个参考行的重建值之间的误差，或者指以两个参考行的重建值为参考，对所述模板区域分别进行预测得到的预测值之间的误差。
  10. 如权利要求1或9所述的方法，其中：
    所述误差用绝对误差和SAD表示，或用误差变换绝对值和SATD表示。
  11. 如权利要求1所述的方法，其中：
    所述当前块的模板区域设置在距离当前块最近的一个参考行上：或者
    所述当前块的模板区域设置在距离当前块最近的多个参考行上。
  12. 一种视频解码方法，包括：
    解码码流中当前块的帧内预测相关语法元素，在当前块允许使用多参考行MRL的情况下，继续解码当前块的多参考行索引，所述多参考行索引用于指示当前块选中的参考行的索引在MRL索引列表中的位置；
    根据解码结果确定允许对当前块的MRL索引列表排序的情况下，按照如权利要求1至11中任一所述的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；
    根据所述多参考行索引和基于当前块排序后的MRL索引列表，确定当前块选中的参考行，根据当前块选中的参考行对当前块进行帧内预测。
  13. 如权利要求12所述的方法，其中：
    所述方法还包括：根据解码结果确定不允许对当前块的MRL索引列表排序的情况下，根据所述多参考行索引和设定的MRL索引列表确定当前块选中的参考行，根据当前块选中的参考行对当前块进行帧内预测。
  14. 如权利要求12所述的方法，其中：
    所述允许对当前块的MRL索引列表排序的充分条件包括：当前块表示允许对MRL索引列表排序的标志为真。
  15. 如权利要求13所述的方法，其中：
    将以下条件中的任意一种或更多种作为不允许对当前块的MRL索引列表排序的充分条件：
    当前块不使用多参考行MRL；
    多参考行索引指示的是MRL索引列表中不参与排序的位置；
    当前帧非亮度帧；
    当前块位于编码树单元CTU的上边界；
    当前块选中的帧内预测模式为指定模式，所述指定模式包括以下模式中的任意一种或更多种：平面模式、有一个模式为平面模式的TIMD融合模式、平均模式、水平角度模式、垂直角度模式、宽角度模式；
    当前块的大小不满足设定条件；
    当前块的长宽比不满足设定条件；
    当前块使用TIMD和当前块不使用TIMD中的一种。
  16. 如权利要求12所述的方法，其中：
    在当前块使用MRL的情况下，所述方法还包括：
    解码当前块能够与MRL同时使用的部分变换模式的语法元素，跳过不能与MRL同时使用的部分变换模式的语法元素；和/或
    在解码当前块选中的帧内预测模式的过程中，允许解码与第二最可能模式MPM相关的语法元素。
  17. 如权利要求12所述的方法，其中：
    所述解码码流中当前块的帧内预测相关语法元素，包括：解码当前块选中的帧内预测模式，以用于对当前块的MRL索引列表排序；
    所述根据当前块选中的参考行对当前块进行帧内预测，包括：根据当前块选中的参考行和当前块选中的帧内预测模式对当前块进行帧内预测。
  18. 如权利要求17所述的方法，其中：
    所述解码当前块的多参考行索引，包括：在当前块使用TIMD的情况下和不使用TIMD的情况下，根据不同的MRL索引列表解码当前块的多参考行索引。
  19. 如权利要求13所述的方法，其中：
    确定当前块不使用TIMD的情况下，所述设定的MRL索引列表为{0,1,3,5,7,12}；
    确定当前块使用TIMD的情况下，所述设定的MRL索引列表为{0,1,3}或{0,1,2}。
  20. 一种视频编码方法，包括：
    对当前块进行帧内预测，确定当前块选中的参考行；
    确定允许对当前块的MRL索引列表排序的情况下，按照如权利要求1至11中任一所述的方法对当前块的MRL索引列表排序，得到基于当前块排序后的MRL索引列表；
    根据当前块选中的参考行的索引在基于当前块排序后的MRL索引列表中的位置，确定当前块的多参考行索引并编码。
  21. 如权利要求20所述的方法，其中：
    所述方法还包括：确定不允许对当前块的MRL索引列表排序的情况下，根据当前块选中的参考行的索引在设定的MRL索引列表中的位置，确定当前块的多参考行索引并编码。
  22. 如权利要求20所述的方法，其中：
    所述方法还包括：确定允许对当前块的MRL索引列表排序的情况下，将当前块表示允许对MRL索引列表排序的标志置为真。
  23. 如权利要求21所述的方法，其中：
    将以下条件中的任意一种或更多种，作为不允许对当前块的MRL索引列表排序的充分条件：
    当前块不使用多参考行MRL；
    当前块选中的参考行位于当前块的模板区域；
    当前帧非亮度帧；
    当前块位于编码树单元CTU的上边界；
    当前块选中的帧内预测模式为指定模式，所述指定模式包括以下模式中的任意一种或更多种：平 面模式、有一个模式为平面模式的TIMD融合模式、平均模式、水平角度模式、垂直角度模式、宽角度模式；
    当前块的大小不满足设定条件；
    当前块的长宽比不满足设定条件；
    当前块使用TIMD和当前块不使用TIMD中的一种；
    在不允许对当前块的MRL索引列表排序的条件均不成立的情况下，确定允许对当前块的MRL索引列表排序。
  24. 如权利要求20所述的方法，其中，所述方法还包括：
    所述确定当前块的多参考行索引并编码，包括：
    在当前块使用TIMD的情况下，根据当前块选中的参考行的索引在第一MRL索引列表中的位置，确定当前块的多参考行索引并编码；
    在当前块不使用TIMD的情况下，根据当前块选中的参考行的索引在第二MRL索引列表中的位置，确定当前块的多参考行索引并编码；其中，所述第二MRL索引列表的长度大于或等于所述第一MRL索引列表的长度。
  25. 如权利要求20所述的方法，其中：
    在当前块使用MRL的情况下，所述方法还包括：编码当前块能够与MRL同时使用的部分变换模式的语法元素，跳过不能与MRL同时使用的部分变换模式的编码。
  26. 如权利要求20所述的方法，其中：
    在当前块使用MRL，且当前块选中的模式非TIMD模式的情况下，所述方法还包括：编码所述多参考行索引之后，继续编码当前块选中的帧内预测模式，且在编码当前块选中的帧内预测模式的过程中，允许编码与第二MPM相关的语法元素。
  27. 如权利要求20所述的方法，其中：
    按照如权利要求1至11中任一所述的方法对当前块的MRL索引列表排序之前，所述方法还包括：确定当前块选中的帧内预测模式。
  28. 一种码流，其中，所述码流中包括当前块的多参考行索引，所述多参考行索引用于指示当前块选中的参考行的索引在MRL索引列表中的位置，所述MRL索引列表是基于当前块排序后的MRL索引列表，或者是设定的MRL索引列表。
  29. 如权利要求28所述的码流，其中：
    所述码流通过如权利要求20至27中任一所述的视频编码方法生成。
  30. 一种多参考行索引列表排序装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如权利要求1至11中任一所述的多参考行索引列表排序方法。
  31. 一种视频解码装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如权利要求12至19中任一所述的视频解码方法。
  32. 一种视频编码装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如权利要求20至27中任一所述的视频编码方法。
  33. 一种视频编解码系统，其中，包括如权利要求32所述的视频编码装置和如权利要求31所述的视频解码装置。
  34. 一种非瞬态计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其中，所述 计算机程序时被处理器执行时能够实现如权利要求1至11中任一所述的多参考行索引列表排序方法，或实现如权利要求12至19中任一所述的视频解码方法，或实现如权利要求20至27中任一所述的视频编码方法。
PCT/CN2022/103944 2022-07-05 2022-07-05 多参考行索引列表排序方法、视频编解码方法、装置和系统 WO2024007157A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2022/103944 WO2024007157A1 (zh) 2022-07-05 2022-07-05 多参考行索引列表排序方法、视频编解码方法、装置和系统
TW112124425A TW202404366A (zh) 2022-07-05 2023-06-29 多參考行索引列表排序方法、視訊編解碼方法、裝置、系統和儲存媒介

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/103944 WO2024007157A1 (zh) 2022-07-05 2022-07-05 多参考行索引列表排序方法、视频编解码方法、装置和系统

Publications (1)

Publication Number Publication Date
WO2024007157A1 true WO2024007157A1 (zh) 2024-01-11

Family

ID=89454731

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/103944 WO2024007157A1 (zh) 2022-07-05 2022-07-05 多参考行索引列表排序方法、视频编解码方法、装置和系统

Country Status (2)

Country Link
TW (1) TW202404366A (zh)
WO (1) WO2024007157A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200014919A1 (en) * 2018-07-05 2020-01-09 Tencent America LLC Methods and apparatus for multiple line intra prediction in video compression
US20200177914A1 (en) * 2018-11-29 2020-06-04 Tencent America LLC Adaptive most probable modes for multiple reference line intra prediction
CN111654696A (zh) * 2020-04-24 2020-09-11 北京大学 一种帧内的多参考行预测方法、装置、存储介质及终端
WO2020256506A1 (ko) * 2019-06-20 2020-12-24 엘지전자 주식회사 다중 참조 라인 인트라 예측을 이용한 영상 부호화/복호화 방법, 장치 및 비트스트림을 전송하는 방법
US20220038684A1 (en) * 2018-10-31 2022-02-03 Interdigital Vc Holdings, Inc. Multi-reference line intra prediction and most probable mode
US20220166968A1 (en) * 2019-03-22 2022-05-26 Lg Electronics Inc. Intra prediction method and apparatus based on multi-reference line in image coding system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200014919A1 (en) * 2018-07-05 2020-01-09 Tencent America LLC Methods and apparatus for multiple line intra prediction in video compression
US20220038684A1 (en) * 2018-10-31 2022-02-03 Interdigital Vc Holdings, Inc. Multi-reference line intra prediction and most probable mode
US20200177914A1 (en) * 2018-11-29 2020-06-04 Tencent America LLC Adaptive most probable modes for multiple reference line intra prediction
US20220166968A1 (en) * 2019-03-22 2022-05-26 Lg Electronics Inc. Intra prediction method and apparatus based on multi-reference line in image coding system
WO2020256506A1 (ko) * 2019-06-20 2020-12-24 엘지전자 주식회사 다중 참조 라인 인트라 예측을 이용한 영상 부호화/복호화 방법, 장치 및 비트스트림을 전송하는 방법
CN111654696A (zh) * 2020-04-24 2020-09-11 北京大学 一种帧内的多参考行预测方法、装置、存储介质及终端

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YAO-JEN CHANG, HONG-JHENG JHU, HUI-YU JIAN, LIANG ZHAO, XIN ZHAO, XIANG LI, SHAN LIU, BENJAMIN BROSS, PAUL KEYDEL, HEIKO SCHWARZ, : "Intra prediction using multiple reference lines for the versatile video coding standard", APPLICATIONS OF DIGITAL IMAGE PROCESSING XLII, SPIE, 6 September 2019 (2019-09-06) - 15 August 2019 (2019-08-15), pages 42, XP055714647, ISBN: 978-1-5106-2968-4, DOI: 10.1117/12.2530564 *

Also Published As

Publication number Publication date
TW202404366A (zh) 2024-01-16

Similar Documents

Publication Publication Date Title
US11729423B2 (en) Apparatus and method for encoding and decoding moving picture using adaptive scanning
CN111149358B (zh) 仿射帧间预测中子块尺寸推导的限制
CN110393010B (zh) 视频译码中的帧内滤波旗标
US8995523B2 (en) Memory efficient context modeling
US20140169452A1 (en) Video encoding method and apparatus using the same
CN112385239A (zh) 亮度块和色度块的分割
US10992964B2 (en) Method and apparatus for determining coding tree node split mode and coding device
US20170134732A1 (en) Systems and methods for digital media communication using syntax planes in hierarchical trees
JP7439841B2 (ja) ループ内フィルタリングの方法及びループ内フィルタリングの装置
US20220201292A1 (en) Adaptive loop filter with fixed filters
CN105306957A (zh) 自适应环路滤波方法和设备
KR20130048180A (ko) 인코딩 및 디코딩을 위한 멀티레벨 유의성 맵
WO2022166462A1 (zh) 编码、解码方法和相关设备
CN110913215B (zh) 一种预测模式的选择方法及装置、可读存储介质
CN114205629B (zh) 一种支持受约束编码树划分的编解码方法及装置
US20060278725A1 (en) Image encoding and decoding method and apparatus, and computer-readable recording medium storing program for executing the method
WO2024007157A1 (zh) 多参考行索引列表排序方法、视频编解码方法、装置和系统
WO2022022299A1 (zh) 视频编解码中的运动信息列表构建方法、装置及设备
WO2024007158A1 (zh) 一种候选列表构建方法、视频编解码方法、装置和系统
WO2024007366A1 (zh) 一种帧内预测融合方法、视频编解码方法、装置和系统
WO2024031691A1 (zh) 一种视频编解码方法、装置和系统
WO2024077576A1 (zh) 基于神经网络的环路滤波、视频编解码方法、装置和系统
WO2024077575A1 (zh) 基于神经网络的环路滤波、视频编解码方法、装置和系统
US20190325615A1 (en) Image encoding/decoding method and apparatus
WO2024077574A1 (zh) 基于神经网络的环路滤波、视频编解码方法、装置和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22949745

Country of ref document: EP

Kind code of ref document: A1