CN111010577B

CN111010577B - Method, device and medium for intra-frame and inter-frame joint prediction in video coding

Info

Publication number: CN111010577B
Application number: CN201911350971.8A
Authority: CN
Inventors: 陈漪纹; 王祥林
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2018-12-31
Filing date: 2019-12-24
Publication date: 2022-03-01
Anticipated expiration: 2039-12-24
Also published as: CN111010577A

Abstract

The invention provides a method and a device for intra-frame and inter-frame joint prediction in video coding, wherein the device comprises the following steps: acquiring a current video image frame, and determining a coding block CU in the video image frame; in the process of decoding a CU by adopting a merging mode, when intra-frame and inter-frame joint prediction CIIP is determined to be adopted, obtaining an inter-frame prediction signal of the CU by utilizing the determined inter-frame prediction mode, and obtaining the intra-frame prediction signal of the CU by utilizing the determined intra-frame prediction mode; and weighting the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal, indicating the determined inter-frame prediction mode to a decoding end through merging indexes, and indicating the code word corresponding to the determined intra-frame prediction mode to the decoding end according to the preset mapping relation between different intra-frame prediction modes and the code word. The method and the device for intra-frame and inter-frame joint prediction in video coding solve the problems of low coding efficiency and complex process of the conventional method for obtaining CIIP prediction.

Description

Method, device and medium for intra-frame and inter-frame joint prediction in video coding

This application claims priority from U.S. patent office filed on 31/12/2018 under application number 62/787, 237, entitled "Intra Mode Coding for the Combined Inter and Intra Prediction," the entire contents of which are incorporated herein by reference.

Technical Field

The present invention relates to the field of video coding and compression technologies, and in particular, to a method, a device, and a medium for intra-frame and inter-frame joint prediction in video coding.

Background

Various video encoding techniques may be used to compress video data when processing the video data. Video encoding is performed according to one or more video encoding standards. For example, Video Coding standards include multifunctional Video Coding (VVC), joint exploration test model (JEM), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), Moving Picture Experts Group (MPEG) Coding, and the like. Video coding typically utilizes prediction methods such as inter prediction, intra prediction, etc., where there is redundancy in the video image or video sequence. An important goal of video coding techniques is to compress video data into a form that uses a lower bit rate while avoiding or minimizing degradation of video quality.

The image partition structure divides the input video into blocks called Coding Tree Units (CTUs). A CTU is divided into a plurality of Coding Units (CUs) using a quad-tree having a nested multi-type tree structure, the coding units having leaf coding units defining regions sharing the same prediction mode. The jvt conference defines the encoding method of VVC. In VVC, when a CU is coded in merge mode and the CU contains at least 64 luma samples, i.e., the CU width x height is equal to or greater than 64, an additional flag is sent to indicate whether Combined Inter and Intra Prediction (CIIP) is applied to the current CU. When CIIP is applied to the current CU, in order to obtain CIIP prediction of the current CU, the Coding end uses a CABAC (Context-adaptive Binary Arithmetic Coding) Coding method to code the related mark information and sends the coded information to the decoding end. The decoding end firstly deduces an intra-frame prediction mode from the information of two bits sent by the encoding end; then, conventional inter and intra decoding processes are used to derive inter and intra prediction signals; finally, the inter and intra prediction signals are weighted averaged to obtain the CIIP prediction. The current CIIP prediction method has the problems of low coding efficiency and complex process.

Disclosure of Invention

The invention provides a method and equipment for intra-frame and inter-frame joint prediction in video coding, which are used for solving the problems of low coding efficiency and complex process of the conventional method for obtaining CIIP prediction.

According to a first aspect of embodiments of the present invention, there is provided a method for intra-frame and inter-frame joint prediction in video coding, the method comprising:

acquiring a current video image frame, and determining a coding block CU in the video image frame;

in the process of decoding the CU in a merging mode, when intra-frame and inter-frame joint prediction CIIP is determined to be adopted, obtaining an inter-frame prediction signal of the CU by using the determined inter-frame prediction mode, and obtaining an intra-frame prediction signal of the CU by using the determined intra-frame prediction mode;

and weighting the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal, indicating the determined inter-frame prediction mode to a decoding end through merging indexes, and indicating the code word corresponding to the determined intra-frame prediction mode to the decoding end according to preset mapping relations between different intra-frame prediction modes and the code word.

According to a second aspect of the embodiments of the present invention, there is provided a method for intra-frame and inter-frame joint prediction in video coding, the method including:

in the process of decoding the CU in a merging mode, when intra-frame and inter-frame joint prediction CIIP is determined to be adopted, determining the inter-frame prediction mode of the current CU according to a merging index indicated by a coding end, and determining the intra-frame prediction mode of the current CU according to the preset mapping relation between different intra-frame prediction modes and code words and the code words indicated by the coding end;

decoding by using the determined inter-frame prediction mode to obtain an inter-frame prediction signal of the CU, and decoding by using the determined intra-frame prediction mode to obtain an intra-frame prediction signal of the CU;

and weighting the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal.

According to a third aspect of the embodiments of the present invention, there is provided an apparatus for intra-frame and inter-frame joint prediction in video coding, including:

the device comprises a coding block acquisition module, a coding block acquisition module and a coding block selection module, wherein the coding block acquisition module is used for acquiring a current video image frame and determining a coding block CU in the video image frame;

the intra-frame and inter-frame prediction module is used for obtaining an inter-frame prediction signal of the CU by using the determined inter-frame prediction mode and obtaining the intra-frame prediction signal of the CU by using the determined intra-frame prediction mode when the CIIP is determined to be adopted in the process of decoding the CU by adopting the merging mode;

and the prediction mode indicating module is used for performing weighting processing on the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal, indicating the determined inter-frame prediction mode to a decoding end through merging indexes, and indicating the code word corresponding to the determined intra-frame prediction mode to the decoding end according to the preset mapping relation between different intra-frame prediction modes and the code word.

Optionally, the indicating module of the prediction mode indicates the codeword corresponding to the determined intra-frame prediction mode to the decoding end according to a preset mapping relationship between different intra-frame prediction modes and the codeword, including:

according to an M-valued process defined in advance according to a coding protocol specification, for code words mapped by different intra-frame prediction modes, indicating M-ary code words corresponding to the determined intra-frame prediction modes to a decoding end through signaling, wherein M is a positive integer greater than 1; or

According to the pre-established corresponding relation between different merging indexes and different intra-frame prediction modes, indicating the determined intra-frame prediction mode to a decoding end through the merging indexes; or

And according to the MPM list obtained by sequencing different intra-frame prediction modes according to the distance between the prediction angles in advance, establishing the mapping relation between different positions in the MPM list and the code words, and indicating the code words corresponding to the positions of the determined intra-frame prediction modes in the MPM list to a decoding end.

Optionally, the mapping, by the prediction mode indication module, the different intra-frame prediction modes to the codewords according to an M-valued process defined by a coding protocol specification includes:

and mapping different intra-frame prediction modes into corresponding binary code words according to a unitary binarization process, a truncated unitary binarization process, a fixed-length binarization process, a truncated Rice binarization process and a K-order exponential Golomb binarization process defined by HEVC (high efficiency video coding) specification.

mapping all intra-frame prediction modes defined by HEVC (high efficiency video coding) specification into corresponding binary codewords according to a binarization process defined by the HEVC specification; or

According to the binarization process defined by the HEVC specification, the mapping relation between the intra-frame prediction mode applied to the CU with the shape and the binary code word is established respectively aiming at different CU shapes.

Optionally, the prediction mode indication module establishes, for different CU shapes, a mapping relationship between the intra prediction mode applied to the CU shape and the binary codeword, including:

the mapping relation between the intra prediction mode and the binary codeword applied to each CU shape is respectively established for CU shapes with the width larger than twice the height, CU shapes with the height larger than twice the width, CU shapes with the width not larger than twice the height and the height not larger than twice the width.

Optionally, before the prediction mode indication module performs weighting processing on the inter-frame prediction signal and the intra-frame prediction signal to obtain the CIIP prediction signal, the method further includes:

if the determined intra-frame prediction mode is a DC prediction mode or a plane prediction mode, or the width or the height of the CU is smaller than N, determining that the inter-frame prediction signal and the intra-frame prediction signal adopt the same weight, wherein N is a positive integer not smaller than 4;

and if the determined intra-frame prediction mode is a horizontal prediction mode or a vertical prediction mode, or the width and the height of the CU are not less than N, determining different weights corresponding to the inter-frame prediction signal and the intra-frame prediction signal according to the determined intra-frame prediction mode and the position of the intra-frame prediction reference sample in the CU.

Optionally, the determining, by the prediction mode indication module, different weights corresponding to the inter prediction signal and the intra prediction signal according to the determined intra prediction mode and the position of the intra prediction reference sample in the CU includes:

if the determined intra-frame prediction mode is a horizontal prediction mode, dividing the CU into n sub-blocks with equal areas in the horizontal direction, and determining n weights from large to small of the corresponding sub-blocks according to the sequence from small to large of the distance from the intra-frame prediction reference sample;

if the determined intra-frame prediction mode is a vertical prediction mode, dividing the CU into n sub-blocks with equal areas in the orthogonal direction, and determining n weights from large to small of the corresponding sub-blocks according to the sequence of the distances from the intra-frame prediction reference samples from small to large, wherein n is a positive integer greater than 1.

Optionally, the indicating module of the prediction mode indicates the codeword corresponding to the determined intra-frame prediction mode to the decoding end, including:

context-adaptive binary arithmetic CABAC coding is carried out on the first m binary bits of the code word corresponding to the determined intra-frame prediction mode, other binary bits are carried out bypass coding and indicated to a decoding end, and m is a positive integer not less than 1, or

And carrying out CABAC coding on the first binary digit of the code word corresponding to the determined intra-frame prediction mode, carrying out bypass coding on other binary digits, and indicating the code digits to a decoding end.

Optionally, the prediction mode indication module is further configured to:

when intra-frame and inter-frame joint prediction CIIP is adopted, if the available intra-frame prediction mode is determined to be only one, the determined inter-frame prediction mode is only indicated to a decoding end through a merging index.

According to a fourth aspect of the embodiments of the present invention, there is provided an apparatus for intra-frame and inter-frame joint prediction in video coding, including:

the prediction mode receiving module is used for determining the inter-frame prediction mode of the current CU according to the merging index indicated by the encoding end when the intra-frame inter-frame joint prediction CIIP is determined to be adopted in the decoding process of adopting the merging mode on the CU, and determining the intra-frame prediction mode of the current CU according to the preset mapping relation between different intra-frame prediction modes and the code words indicated by the encoding end;

the intra-frame prediction module is used for decoding by utilizing the determined inter-frame prediction mode to obtain an inter-frame prediction signal of the CU and decoding by utilizing the determined intra-frame prediction mode to obtain an intra-frame prediction signal of the CU;

and the prediction signal determining module is used for weighting the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal.

Optionally, the determining, by the prediction mode receiving module, the intra prediction mode of the current CU according to a preset mapping relationship between different intra prediction modes and codewords and the codewords indicated by the encoding end includes:

according to an M-valued process defined in advance according to a coding protocol specification, determining that an intra-frame prediction mode corresponding to an M-ary code word indicated by a coding end through signaling is an intra-frame prediction mode of a current CU for the M-ary code word mapped by different intra-frame prediction modes, wherein M is a positive integer greater than 1; or

Determining the intra-frame prediction mode corresponding to the merging index indicated by the encoding end as the intra-frame prediction mode of the current CU according to the pre-established corresponding relation between different merging indexes and different intra-frame prediction modes; or

And determining a most probable mode MPM list according to the distance between the prediction angles in advance, establishing a mapping relation between different positions and code words in the MPM list, and determining that the intra-frame prediction mode of the code word indicated by the encoding end at the corresponding position in the MPM list is the intra-frame prediction mode of the current CU.

Optionally, the mapping, by the prediction mode receiving module, the different intra-frame prediction modes to the codewords according to an M-valued process defined by a coding protocol specification includes:

Optionally, the prediction mode receiving module establishes, for different CU shapes, a mapping relationship between the intra prediction mode applied to the CU shape and the binary codeword, including:

the mapping relation between the intra prediction mode and the binary codeword applied to each CU shape is respectively established for CU shapes with the width of the CU larger than twice the height, CU shapes with the height of the CU larger than twice the width, CU shapes with the width of the CU not larger than twice the height and the height not larger than twice the width.

Optionally, before the predicting signal determining module performs weighting processing on the inter-frame prediction signal and the intra-frame prediction signal to obtain the CIIP prediction signal, the method further includes:

Optionally, the determining the intra prediction mode and the position of the intra prediction reference sample in the CU, and determining the inter prediction signal and the intra prediction signal corresponding to different weights includes:

if the determined intra-frame prediction mode is a vertical prediction mode, dividing the CU into n sub-blocks with equal areas in the orthogonal direction, and determining n weights from large to small of the corresponding sub-blocks according to the sequence of the distances from the intra-frame prediction reference samples from small to large, wherein n is a positive integer larger than 1.

Optionally, the determining, by the prediction mode receiving module, the intra prediction mode of the current CU according to the codeword indicated by the encoding end includes:

and decoding the coded code word indicated by the coding end to obtain a corresponding code word, and determining that the intra-frame prediction mode corresponding to the code word is the intra-frame prediction mode of the current CU.

Optionally, the prediction mode receiving module is further configured to:

when intra-frame inter-frame joint prediction CIIP is adopted, when the available intra-frame prediction mode is determined to be only one, the intra-frame prediction mode is determined to be the intra-frame prediction mode of the current CU.

According to a fifth aspect of the embodiments of the present invention, there is provided an apparatus for intra-frame and inter-frame joint prediction in video coding, including: a memory and a processor;

wherein the memory is used for storing programs;

the processor is used for executing the program in the memory and comprises the following steps:

Optionally, the indicating, by the processor, the codeword corresponding to the determined intra-frame prediction mode to the decoding end according to a preset mapping relationship between different intra-frame prediction modes and the codeword includes:

Optionally, the processor maps the different intra prediction modes to the codeword according to an M-valued process defined by a coding protocol specification, including:

Optionally, before the processor performs weighting processing on the inter-frame prediction signal and the intra-frame prediction signal to obtain the CIIP prediction signal, the method further includes:

Optionally, the processor determines different weights corresponding to the inter prediction signal and the intra prediction signal according to the determined intra prediction mode and the position of the intra prediction reference sample in the CU, and includes:

Optionally, the processor indicates the codeword corresponding to the determined intra prediction mode to the decoding end, including:

Optionally, the processor is further configured to:

According to a sixth aspect of the embodiments of the present invention, there is provided an apparatus for intra-frame and inter-frame joint prediction in video coding, including: a memory and a processor;

wherein the memory is used for storing programs;

Optionally, the determining, by the processor, the intra prediction mode of the current CU according to a preset mapping relationship between different intra prediction modes and codewords and the codewords indicated by the encoding end includes:

Optionally, the processor establishes, for different CU shapes, a mapping relationship between the intra prediction mode applied to the CU in the shape and the binary codeword, including:

Optionally, the processor determines the intra prediction mode of the current CU according to the codeword indicated by the encoding end, including:

Optionally, the processor is further configured to:

According to a seventh aspect of the embodiments of the present invention, there is provided a chip, the chip is coupled with a memory in a device, so that the chip invokes program instructions stored in the memory when running, thereby implementing the above aspects of the embodiments of the present application and any method that may be designed according to the aspects.

According to an eighth aspect of the embodiments of the present invention, there is provided a computer-readable storage medium storing program instructions which, when executed on a computer, cause the computer to perform the method of any of the possible designs to which the above aspects and aspects relate.

According to a ninth aspect of the embodiments of the present invention, there is provided a computer program product, which, when run on an electronic device, causes the electronic device to perform a method of implementing the above aspects of the embodiments of the present application and any possible design related to the aspects.

The method and the equipment for intra-frame and inter-frame joint prediction in video coding have the following beneficial effects:

according to the intra-frame and inter-frame joint prediction method and device in video coding, the coding end sends the indication about the intra-frame prediction mode in the CIIP prediction mode to the decoding end, the decoding end determines the intra-frame prediction mode when CIIP prediction is adopted according to the indication of the coding end, and performs weighting processing on intra-frame and inter-frame signals according to the intra-frame prediction mode to obtain CIIP prediction signals, so that the problems that the existing method for obtaining CIIP prediction is low in coding efficiency and complex in process are solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a diagram illustrating a method for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention;

FIG. 2 is a diagram of exemplary binary codewords obtained by different binarization methods according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a binary codeword generated by truncated rice binarization according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a binary codeword generated by fixed-length binarization according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating a method for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention;

FIG. 6 is a block diagram of an apparatus for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention;

FIG. 7 is a block diagram of an apparatus for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention;

FIG. 8 is a block diagram illustrating an apparatus for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an apparatus for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

For convenience of understanding, terms referred to in the embodiments of the present invention are explained below:

1) CABAC Coding (Context-adaptive Binary Arithmetic Coding): is a form of entropy coding used in coding standards such as high-efficiency video coding, VVC, etc.; CABAC, which has several innovations and modifications to adapt to the requirements of video coding standards based on arithmetic coding, has multiple probability modes for different contexts, first converts all non-binary symbols into binaries, then selects a probability model to be used for each binary (also called bit), optimizes probability estimates, and finally compresses the data with arithmetic coding;

2) and (3) binarization encoding: the method refers to a process of encoding non-binary characters into binary strings according to a certain rule during arithmetic coding in video coding, and common binary coding algorithms include unitary binarization, truncated unitary binarization, fixed-length binarization, truncated rice binarization, K-order exponential Golomb binarization and the like.

Example 1

The embodiment of the present invention provides a method for intra-frame and inter-frame joint prediction in video coding, which is applied to a video coding end, as shown in fig. 1, and includes:

step S101, acquiring a current video image frame, and determining a coding block CU in the video image frame;

acquiring a current video image frame, and determining a coding block CU in the video image frame. The image division structure divides an input current video image frame into blocks called coding tree units CTUs, and divides one coding tree unit CTU into coding blocks CU using a quad tree having a nested multi-type tree structure, the coding blocks CU having leaf coding units defining regions sharing the same prediction mode, which may be intra-frame or inter-frame, etc. Each CU includes one or more prediction units PU and a transform unit tree.

Step S102, in the process of decoding the CU in the merging mode, when the CIIP (joint prediction in intra-frame and inter-frame) is determined to be adopted, the inter-frame prediction signal of the CU is obtained by using the determined inter-frame prediction mode, and the intra-frame prediction signal of the CU is obtained by using the determined intra-frame prediction mode;

the coding end has six available interframe prediction modes, one of which is selected for prediction when interframe signal prediction is carried out, and has four available intraframe prediction modes, one of which is selected for prediction when intraframe signal prediction is carried out.

In the process of decoding a CU by adopting a merging mode, when intra-frame and inter-frame joint prediction CIIP is determined, the inter-frame prediction signal of the CU is determined according to the inter-frame prediction mode used by a coding end, and the intra-frame prediction signal of the CU is determined according to the intra-frame prediction mode used by the coding end. In particular implementation, in inter prediction, an inter prediction value may be formed by motion estimation and motion compensation based on pixels from a previous reconstructed frame; in intra prediction, an intra prediction value may be formed based on reconstructed pixels in a current frame.

With CIIP prediction mode, the inter prediction signal is derived using the same inter prediction process as applied to the conventional merge mode, and the intra prediction signal is derived using CIIP intra prediction mode after the conventional intra prediction process.

The above method for deriving the inter prediction signal and the intra prediction signal in the CIIP prediction mode can adopt the prior art, and is not described in detail here.

Step S103, weighting the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal, indicating the determined inter-frame prediction mode to a decoding end through merging indexes, and indicating the code word corresponding to the determined intra-frame prediction mode to the decoding end according to the preset mapping relation between different intra-frame prediction modes and the code word.

And performing weighted average on the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal, wherein the weighted value depends on the used intra-frame prediction mode or the size of the CU, and the weighted value comprises the following two conditions:

1) and if the determined intra-frame prediction mode is a DC prediction mode or a plane prediction mode, or the width or the height of the current CU is less than N, determining that the inter-frame prediction signal and the intra-frame prediction signal adopt the same weight, wherein N is a positive integer not less than 4.

2) And if the determined intra-frame prediction mode is a horizontal prediction mode or a vertical prediction mode, or the width and the height of the current CU are not less than N, determining different weights corresponding to the inter-frame prediction signal and the intra-frame prediction signal according to the determined intra-frame prediction mode and the position of the intra-frame prediction reference sample in the CU.

In specific implementation, if the determined intra-frame prediction mode is a horizontal prediction mode, dividing a CU into n sub-blocks with equal areas in the horizontal direction, and determining n weights from large to small of the corresponding sub-blocks according to the sequence from small to large of the distance from the intra-frame prediction reference sample;

if the determined intra-frame prediction mode is a vertical prediction mode, dividing the CU into n sub-blocks with equal areas in the orthogonal direction, and determining n weights from large to small of the corresponding sub-blocks according to the sequence from small to large of the distance from the intra-frame prediction reference sample, wherein n is a positive integer larger than 1.

Taking the determined intra prediction mode as the horizontal prediction mode as an example, let W be the width of the current block and H be the height of the current block, the coding block is first divided into four equal-area portions in the horizontal direction, the size (W/4) x H of each portion, the weight wt of each of the four regions is set to 6, 5, 3 and 2 from the portion closest to the intra prediction reference sample to the portion farthest from the intra prediction reference sample, assuming that the determined inter prediction signal is represented as P_interThe determined inter prediction signal is denoted as P_intraThe final CIIP prediction signal P_CIIPCan be derived by the following formula:

P_CIIP＝((8-wt)*P_inter+wt*P_intra)＞＞3

and presetting the mapping relation between different intra-frame prediction modes and the code words at the encoding end, so as to indicate the code words corresponding to the determined intra-frame prediction modes to the decoding end. Presetting the mapping relation between different intra-frame prediction modes and the code words, and adopting any one of the following methods:

1) according to an M-valued process defined by the HEVC specification, different intra prediction modes are mapped to corresponding M-ary codewords, where M is a positive integer greater than 1.

The intra-frame prediction modes commonly used in the CIIP prediction include four modes, namely a DC prediction mode, a plane prediction mode, a horizontal prediction mode and a vertical prediction mode. The shape of the CU includes a CU shape having a width greater than twice the height, a CU shape having a height greater than twice the width, a CU shape having a width not greater than twice the height and a height not greater than twice the width. In the case where the shape of a CU is a CU shape whose width is more than twice the height, the horizontal prediction mode is not allowed to be used, and in the case where the shape of a CU is a CU shape whose height is more than twice the width, the vertical prediction mode is not allowed to be used, in which case, only three prediction modes are allowed to be used.

When different intra-frame prediction modes are mapped into corresponding M-ary code words according to an M-valued process defined by HEVC (high efficiency video coding) specifications, a binarization process can be selected, and all intra-frame prediction modes defined by the HEVC specifications are mapped into corresponding binary code words according to the binarization process defined by the HEVC specifications; or according to the binarization process defined by the HEVC specification, mapping relationships between intra prediction modes and binary codewords applied to the CU of the shape are established for different CU shapes, respectively, that is, mapping relationships between intra prediction modes and binary codewords applied to the CU shapes are established for CU shapes having a width greater than twice the height, CU shapes having a height greater than twice the width, and CU shapes having a width not greater than twice the height and a height not greater than twice the width.

In specific implementation, different intra-frame prediction modes are mapped into corresponding binary codewords according to any one of a unitary binarization process, a truncated unitary binarization process, a fixed-length binarization process, a truncated rice binarization process and a K-order exponential golomb binarization process defined by the HEVC specification. The binarization process is defined in the specification, and the prior art can be adopted when the binarization process is implemented, and the details are not described here.

Referring to fig. 2, a schematic diagram of exemplary binary codewords obtained by using different binarization methods according to an embodiment of the present invention is provided. Assuming that there are N +1 intra prediction modes from mode 0 to mode N, the binary codeword for each intra prediction mode is generated using the same binarization process. For example, the correspondence between a column of binary codewords corresponding to truncated rice binarization and N +1 intra prediction modes as shown in fig. 2 constitutes a mapping between the intra prediction modes and the binary codewords generated according to the truncated rice binarization process defined in the HEVC specification. In the binarization process, "0" and "1" assigned to the binary codeword may be used in reverse, for example, in fig. 2, the binary codeword of mode 0 generated by unary binarization is "0", the binary codeword of mode 5 is "111110", and when used in reverse, the binary codeword of mode 0 is "1", and the binary codeword of mode 5 is "000001".

Referring to fig. 3, a schematic diagram of a binary codeword generated by truncated rice binarization according to an embodiment of the present invention is shown. Fig. 4 is a schematic diagram of a binary codeword generated by fixed-length binarization according to an embodiment of the present invention. Assuming that a CU shape having a width greater than twice the height is a first CU shape, a CU shape having a height greater than twice the width is a second CU shape, and a CU shape having a width not greater than twice the height and a height not greater than twice the width is a third CU shape, mapping relationships between the intra prediction mode applied to the CU shape and the binary codeword, which are established for different CU shapes, are given in fig. 3, respectively, and are shown in (a), (b), (c), and (d) parts of the figure. Fig. 4 shows mapping relationships between four intra-prediction modes and binary codewords generated by using a fixed-length binarization process, where the mapping relationships between intra-prediction modes and binary codewords applied to a CU in a shape are established for different CU shapes, and are respectively shown in (a), (b), (c), and (d) in the figure. According to the shape of the current CU and the determined intra prediction mode, by looking up the mapping relationship table between the preset intra prediction mode and the binary codeword as shown in fig. 3 or fig. 4, the encoding end can determine the corresponding binary codeword, then encode the binary codeword, and send the encoded binary codeword to the decoding end, thereby instructing the decoding end to determine the corresponding intra prediction mode.

The mapping relationship shown in fig. 3 or fig. 4 is only described as an example, and other binarization processes may be used to generate the corresponding mapping relationship in practice, and will not be described in detail here.

2) And establishing corresponding relations between different merging indexes and different intra-frame prediction modes.

The intra prediction mode is signaled together with the merge index by establishing correspondence of different merge indices and different intra prediction modes.

As an alternative, the merge index needs to indicate six inter prediction modes, and needs to indicate four intra prediction modes, and there are 6 combinations of currently available intra prediction modes and inter prediction modes. As a possible implementation, the planar prediction mode is corresponding to the

merge indices

0 and 1, the DC prediction mode is corresponding to the

merge indices

2 and 3, the horizontal prediction mode is corresponding to the merge index 4, and the vertical prediction mode is corresponding to the merge index 5.

As another possible implementation, the vertical prediction mode is assigned to the merge index 0, the horizontal prediction mode is assigned to the merge index 1, the planar prediction mode is assigned to the

merge indices

2 and 3, and the DC prediction mode is assigned to the

merge indices

4 and 5.

When intra-frame and inter-frame joint prediction CIIP is adopted, if the available intra-frame prediction mode is determined to be only one, the determined inter-frame prediction mode is only indicated to a decoding end through a merging index. As an alternative embodiment, if it is determined that only the plane prediction mode can be used, merging indexes 0-5 are all corresponding to the plane prediction mode.

The above two corresponding manners are only exemplary, and other methods for combining the intra prediction mode and the merge index may be included when actually determining the corresponding relationship.

3) And establishing mapping relations between different positions in the MPM list and the code words according to the most possible mode MPM list determined by the distance between the prediction angles.

Generating an MPM list according to an MPM list generation mode of a general intra-frame prediction mode, wherein the MPM list comprises all possible 67 intra-frame prediction modes, 65 intra-frame prediction modes except a DC prediction mode and a plane prediction mode have different prediction angles, according to the prediction angle difference between the prediction angle and the horizontal prediction mode and the vertical prediction mode, the prediction mode corresponding to the prediction angle is corresponding to the horizontal prediction mode or the vertical prediction mode, finally generating a corresponding MPM list, and then removing the redundant prediction mode in the MPM list to determine a final MPM list, wherein the final MPM list comprises the DC prediction mode, the plane prediction mode, the horizontal prediction mode and the vertical prediction mode. For example, if an angle mode in the conventional MPM list is closer to a vertical prediction mode, it is corresponding to the vertical prediction mode.

The multiple intra-frame prediction modes in the MPM list are ordered according to the most probable intra-frame prediction mode, the encoding end selects one intra-frame prediction mode from the multiple intra-frame prediction modes, and the position of the selected intra-frame prediction mode in the MPM list is indicated to the decoding end.

The above-mentioned MPM list generation method for the general intra prediction mode is a formation method of an MPM candidate list in the conventional CIIP prediction, and the MPM list generation in the present embodiment can adopt the conventional technique, and is not described in detail here.

The method includes that a mapping relation between different intra-frame prediction modes and code words is preset by any method, when CIIP prediction is carried out, a coding end indicates the determined inter-frame prediction mode to a decoding end through merging indexes, and indicates the code words corresponding to the determined intra-frame prediction mode to the decoding end, and the method specifically includes the following steps:

when the method 1) is adopted to preset the mapping relation between different intra-frame prediction modes and code words, the M-ary code words mapped by the different intra-frame prediction modes are indicated to a decoding end through signaling according to the M-valued process defined in advance according to the HEVC specification;

when the mapping relation between different intra-frame prediction modes and the code words is preset by adopting the mode 2), the determined intra-frame prediction mode is indicated to a decoding end through the merging index according to the corresponding relation between different pre-established merging indexes and different intra-frame prediction modes;

when the mapping relation between different intra-frame prediction modes and the code words is preset in the mode 3), the mapping relation between different positions in the MPM list and the code words is established according to the most probable mode MPM list determined in advance according to the distance between the prediction angles, and the code words corresponding to the determined positions of the intra-frame prediction modes in the MPM list are indicated to a decoding end.

And when the encoding end indicates the code word corresponding to the determined intra-frame prediction mode to the decoding end, the code word corresponding to the determined intra-frame prediction mode is encoded by adopting a CABAC (context-based adaptive binary arithmetic coding) method. Non-binary symbols are first "binarized" or converted to binary code, and each binary digit (or bit) of the binarized symbol is then encoded. Different context models may be selected when encoding, a context model being a probability model for one or more bins of a binarized symbol, the probability of each bin being stored. During encoding, an arithmetic encoder at the encoding end encodes each binary bit according to the selected context probability model.

As a possible implementation, context adaptive binary arithmetic CABAC coding is performed on the first m bins of the codeword corresponding to the determined intra prediction mode, each bin may have a context, and the other bins except the first N bins are bypass-coded, where m is a positive integer not less than 1. And after the coding is finished, the coding end indicates the coded code words to the decoding end.

As another possible implementation, the first binary bit of the codeword corresponding to the determined intra prediction mode is CABAC encoded, the coding model is selected according to the coding information, e.g., the size of the CU, and the other binary bits are bypass encoded. And after the coding is finished, the coding end indicates the coded code words to the decoding end.

The embodiment of the invention also provides a method for intra-frame and inter-frame joint prediction in video coding, which is applied to a video decoding end. As shown in fig. 5, the method includes:

step S501, acquiring a current video image frame, and determining a coding block CU in the video image frame;

and acquiring a current video image frame needing to be processed, and determining a coding block CU. The image division structure divides an input current video image frame into blocks called coding tree units CTUs, and divides one coding tree unit CTU into coding blocks CU using a quad tree having a nested multi-type tree structure, the coding blocks CU having leaf coding units defining regions sharing the same prediction mode, which may be intra-frame or inter-frame, etc. Each CU includes one or more prediction units PU and a transform unit tree.

Step S502, in the process of decoding the CU in the merging mode, when the CIIP is determined to be adopted in intra-frame and inter-frame joint prediction, determining the inter-frame prediction mode of the current CU according to the merging index indicated by the encoding end, and determining the intra-frame prediction mode of the current CU according to the preset mapping relation between different intra-frame prediction modes and the code words indicated by the encoding end;

the mapping relation between different intra-frame prediction modes and the code words is preset at a decoding end, so that after the indication of an encoding end is received, the corresponding intra-frame prediction mode is determined. When the mapping relation between different intra-frame prediction modes and code words is preset, any one of the following methods is adopted:

1) and mapping different intra-frame prediction modes into code words according to an M-valued process defined by a coding protocol specification.

According to an M-valued process defined by a coding protocol specification, different intra-frame prediction modes are mapped into code words, and all intra-frame prediction modes defined by the HEVC specification can be mapped into corresponding binary code words according to a binarization process defined by the HEVC specification; or, according to the binarization process defined in the HEVC specification, mapping relationships between intra prediction modes and binary codewords applied to the CU of the shape are established for different CU shapes, respectively, that is, mapping relationships between intra prediction modes and binary codewords applied to the CU shapes are established for CU shapes whose width is greater than twice the height, CU shapes whose height is greater than twice the width, and CU shapes whose width is not greater than twice the height and whose height is not greater than twice the width.

Specifically, any one of a unitary binarization process, a truncated unitary binarization process, a fixed-length binarization process, a truncated rice binarization process and a K-order exponential golomb binarization process defined by the HEVC specification may be adopted to map different intra-frame prediction modes into corresponding binary codewords. In practice, the same method as the above decoding end is adopted, and detailed description is omitted here.

In practice, the same method as the above decoding end is adopted, and detailed description is omitted here.

Receiving an indication of an encoding end, determining an inter-frame prediction mode of a current CU according to a merging index indicated by the encoding end, decoding an encoded codeword indicated by the encoding end to obtain a corresponding codeword, and determining an intra-frame prediction mode corresponding to the codeword as the intra-frame prediction mode of the current CU according to a preset mapping relationship between different intra-frame prediction modes and the codeword, wherein the method specifically comprises the following steps:

when the method 1) is adopted to preset the mapping relationship between different intra-frame prediction modes and the code words, according to an M-valued process defined in advance according to the coding protocol specification, for the M-ary code words mapped by the different intra-frame prediction modes, it is determined that the intra-frame prediction mode corresponding to the M-ary code word indicated by the coding end through signaling is the intra-frame prediction mode of the current CU, where M is a positive integer greater than 1.

When the mapping relationship between different intra-frame prediction modes and the code words is preset in the mode 2), the intra-frame prediction mode corresponding to the merging index indicated by the encoding end is determined to be the intra-frame prediction mode of the current CU according to the pre-established corresponding relationship between different merging indexes and different intra-frame prediction modes.

When the above method 3) is adopted to preset the mapping relationship between different intra-frame prediction modes and codewords, determining a most probable mode MPM list according to the distance between prediction angles in advance, establishing the mapping relationship between different positions and codewords in the MPM list, and determining the intra-frame prediction mode of the codeword indicated by the encoding end at the corresponding position in the MPM list as the intra-frame prediction mode of the current CU.

When intra-frame and inter-frame joint prediction CIIP is adopted, if the available intra-frame prediction mode is determined to be only one, the decoding end determines that the intra-frame prediction mode is the intra-frame prediction mode of the current CU, and determines the inter-frame prediction mode of the current CU according to the merging index indicated by the decoding end.

Step S503, using the determined inter-frame prediction mode to decode to obtain the inter-frame prediction signal of the CU, and using the determined intra-frame prediction mode to decode to obtain the intra-frame prediction signal of the CU;

and after the conventional intra-frame prediction process, the intra-frame prediction mode determined in the step is used for deriving and obtaining the intra-frame prediction signal of the current CU.

The above-mentioned method for determining the inter prediction signal and the intra prediction signal can adopt the prior art, and will not be described in detail here.

Step S504, weighting the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal.

And carrying out weighted average on the inter-frame prediction signal and the intra-frame prediction signal obtained in the step to obtain a CIIP prediction signal.

Wherein, the weight value depends on the intra prediction mode, including the following two cases:

In the method for intra-frame and inter-frame joint prediction in video coding provided by the above embodiment, the same mapping relationship between the intra-frame prediction mode and the codeword is preset at the encoding end and the decoding end, an indication about the intra-frame prediction mode in the CIIP prediction mode is sent to the decoding end through the encoding end, the decoding end determines the intra-frame prediction mode when the CIIP prediction is adopted according to the indication at the encoding end, and performs weighting processing on intra-frame and inter-frame signals according to the intra-frame prediction mode to obtain CIIP prediction signals, so that the problems of low encoding efficiency and complex process of the existing method for obtaining the CIIP prediction are solved.

Example 2

The method for intra-frame and inter-frame joint prediction in video coding according to the present invention is described above, and an apparatus for performing the method for intra-frame and inter-frame joint prediction in video coding is described below.

Referring to fig. 6, an embodiment of the present invention provides an apparatus for intra-frame and inter-frame joint prediction in video coding, the apparatus being applied to a video decoding end, and the apparatus comprising:

a coding block obtaining module 601, configured to obtain a current video image frame and determine a coding block CU in the video image frame;

an intra-frame prediction module 602, configured to, when it is determined that intra-frame inter-frame joint prediction CIIP is used in a merging mode decoding process for the CU, obtain an inter-frame prediction signal of the CU using the determined inter-frame prediction mode, and obtain an intra-frame prediction signal of the CU using the determined intra-frame prediction mode;

the prediction mode indication module 603 is configured to perform weighting processing on the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal, indicate the determined inter-frame prediction mode to the decoding end through merging indexes, and indicate a codeword corresponding to the determined intra-frame prediction mode to the decoding end according to a preset mapping relationship between different intra-frame prediction modes and the codeword.

Optionally, the prediction mode indication module is further configured to:

Referring to fig. 7, an embodiment of the present invention provides an apparatus for intra-frame and inter-frame joint prediction in video coding, the apparatus being applied to a video decoding end, and the apparatus comprising:

a coding block obtaining module 701, configured to obtain a current video image frame and determine a coding block CU in the video image frame;

a prediction mode receiving module 702, configured to, when it is determined that intra-frame inter-frame joint prediction CIIP is used in a merging mode decoding process for the CU, determine an inter-frame prediction mode of the current CU according to a merging index indicated by a coding end, and determine an intra-frame prediction mode of the current CU according to a preset mapping relationship between different intra-frame prediction modes and codewords indicated by the coding end;

an intra-frame prediction module 703, configured to decode using the determined inter-frame prediction mode to obtain an inter-frame prediction signal of the CU, and decode using the determined intra-frame prediction mode to obtain an intra-frame prediction signal of the CU;

a prediction signal determining module 704, configured to perform weighting processing on the inter-frame prediction signal and the intra-frame prediction signal to obtain a CIIP prediction signal.

Optionally, the prediction mode receiving module is further configured to:

The apparatus for intra-frame and inter-frame joint prediction in video coding in the embodiment of the present application is described above from the perspective of a modular functional entity, and the apparatus for intra-frame and inter-frame joint prediction in video coding in the embodiment of the present application is described below from the perspective of hardware processing.

Referring to fig. 8, another embodiment of the apparatus for intra-frame and inter-frame joint prediction in video coding according to the embodiment of the present application includes:

a processor 801, a memory 802, a transceiver 809, and a bus system 811;

wherein the memory is used for storing programs;

Fig. 8 is a schematic structural diagram of an apparatus for intra-frame and inter-frame joint prediction in video coding according to an embodiment of the present invention, which is applied to a video coding end, where the apparatus 800 may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPU) 801 (e.g., one or more processors) and a memory 802, and one or more storage media 803 (e.g., one or more mass storage devices) for storing applications 804 or data 806. Memory 802 and storage medium 803 may be, among other things, transient storage or persistent storage. The program stored in the storage medium 803 may include one or more modules (not shown), and each module may include a series of instruction operations for the information processing apparatus. Still further, the processor 801 may be configured to communicate with the storage medium 803 to execute a series of instruction operations in the storage medium 803 on the device 800.

The apparatus 800 may also include one or more power supplies 810, one or more wired or wireless network interfaces 807, one or more input-output interfaces 808, and/or one or more operating systems 805, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

Optionally, the processor is further configured to:

Referring to fig. 9, another embodiment of the apparatus for intra-frame and inter-frame joint prediction in video coding according to the embodiment of the present application includes:

a processor 901, a memory 902, a transceiver 909, and a bus system 911;

wherein the memory is used for storing programs;

Fig. 9 is a schematic structural diagram of an apparatus for intra-frame and inter-frame joint prediction in video coding, which is applied to a video decoding end, and the apparatus 900 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPU) 901 (e.g., one or more processors) and a memory 902, and one or more storage media 903 (e.g., one or more mass storage devices) storing an application 904 or data 906. Memory 902 and storage medium 903 may be, among other things, transient storage or persistent storage. The program stored in the storage medium 903 may include one or more modules (not shown), and each module may include a series of instruction operations in the information processing apparatus. Further, the processor 901 may be configured to communicate with the storage medium 903 and execute a series of instruction operations in the storage medium 903 on the device 900.

Device 900 may also include one or more power supplies 910, one or more wired or wireless network interfaces 907, one or more input-output interfaces 908, and/or one or more operating systems 905, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

Optionally, the processor is further configured to:

Embodiments of the present invention also provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to perform the method for intra-frame and inter-frame joint prediction in video coding provided by the foregoing embodiments.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

The technical solutions provided by the present application are introduced in detail, and the present application applies specific examples to explain the principles and embodiments of the present application, and the descriptions of the above examples are only used to help understand the method and the core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method for intra-frame and inter-frame joint prediction in video coding, comprising:

if the determined intra-frame prediction mode is a horizontal prediction mode or a vertical prediction mode, or the width and the height of the CU are not less than N, determining different weights corresponding to the inter-frame prediction signal and the intra-frame prediction signal according to the determined intra-frame prediction mode and the position of an intra-frame prediction reference sample in the CU;

2. The method of claim 1, wherein indicating the codeword corresponding to the determined intra-frame prediction mode to a decoding end according to a mapping relationship between preset different intra-frame prediction modes and the codeword comprises:

3. The method of claim 2, wherein mapping different intra prediction modes to codewords according to an M-valued process defined by a coding protocol specification comprises:

4. The method of claim 2, wherein mapping different intra prediction modes to codewords according to an M-valued process defined by a coding protocol specification comprises:

5. The method of claim 4, wherein the mapping relationship between the intra prediction mode and the binary codeword applied to the CU in the shape is established for different CU shapes respectively, and comprises:

6. The method of claim 1, wherein determining different weights for the inter-prediction signal and the intra-prediction signal according to the determined intra-prediction mode and the position of the intra-prediction reference sample in the CU comprises:

7. The method of claim 1, wherein indicating the codeword corresponding to the determined intra prediction mode to a decoding end comprises:

8. The method of claim 1, further comprising:

9. A method for intra-frame and inter-frame joint prediction in video coding, comprising:

10. The method of claim 9, wherein determining the intra prediction mode of the current CU according to a predetermined mapping relationship between different intra prediction modes and codewords indicated by a coding end comprises:

11. The method of claim 10, wherein mapping different intra prediction modes to codewords according to an M-valued process defined by a coding protocol specification comprises:

12. The method of claim 10, wherein mapping different intra prediction modes to codewords according to an M-valued process defined by a coding protocol specification comprises:

13. The method of claim 12, wherein the mapping relationship between the intra prediction mode and the binary codeword applied to the CU of the shape is established for different CU shapes respectively, and comprises:

14. The method of claim 9, wherein determining different weights for the inter-prediction signal and the intra-prediction signal according to the determined intra-prediction mode and the position of the intra-prediction reference sample in the CU comprises:

15. The method of claim 9, wherein determining the intra prediction mode of the current CU according to the codeword indicated by the encoding end comprises:

16. The method of claim 9, further comprising:

17. An apparatus for intra-frame and inter-frame joint prediction in video coding, comprising: a memory and a processor;

wherein the memory is for storing a computer program;

the processor is used for executing the program in the memory and realizing the steps of the method according to any one of claims 1 to 8.

18. An apparatus for intra-frame and inter-frame joint prediction in video coding, comprising: a memory and a processor;

wherein the memory is for storing a computer program;

the processor is used for executing the program in the memory and realizing the steps of the method according to any one of claims 9-16.

19. A computer program medium, having a computer program stored thereon, wherein the program, when executed by a processor, performs the steps of the method according to any one of claims 1 to 8, or performs the steps of the method according to any one of claims 9 to 16.