RU2785725C2 - Device and method for external prediction - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: group of inventions relates to the field of video encoding technologies, in particular to a method and a device for external prediction. A method for external prediction is proposed. The method contains a stage, at which a bit stream is syntactically analyzed to obtain information of movement of an image block to be processed. Next, according to the method, movement for the image block to be processed is compensated based on movement information to obtain a prediction block of the image block to be processed, in which the prediction block of the image block to be processed contains a value of prediction of a target sample. Weighing for one or several restored values of one or several reference samples and values of prediction of the target sample are calculated to update values of prediction of the target sample, in which the target sample has a set relation of a spatial position to the target sample.

EFFECT: increase in the efficiency of prediction due to elimination of time redundancy and spatial redundancy in a video.

24 cl, 19 dwg, 5 tbl

Description

The technical field to which the invention belongs

Embodiments of the present invention relate to the field of video coding technologies and, in particular, to an inter prediction method and apparatus.

State of the art

Digital video technologies can be widely used in various digital video devices. The digital video device may implement video coding technologies, such as the standards defined in MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4 Part 10 Advanced Video Coding (AVC), ITU-T T H.265 (also called HEVC High Efficiency Video Coding) and the video coding technologies described in extensions to these standards. A digital video device sends, receives, encodes, decodes, and/or stores digital video information more efficiently by implementing these video coding technologies.

At present, inter prediction and intra prediction technologies are mainly used in video coding to eliminate temporal redundancy and spatial redundancy in video. The inter-prediction technology takes into account only the temporal correlation between the same objects in adjacent image frames, but the spatial correlation is not taken into account. Therefore, the prediction samples obtained in the existing inter prediction mode are spatially discontinuous. In addition, the prediction performance is affected, and the residual prediction energy is relatively high.

The essence of the invention

Embodiments of the present invention provide an inter prediction method and apparatus for performing spatial filtering of an inter-coded prediction block and improving coding efficiency.

To solve technical problems in the embodiments of the present invention, the following technical solutions are used.

According to a first aspect of the embodiments of the present invention, an inter prediction method is provided. The method includes: parsing the bit stream to obtain motion information of the image block to be processed; performing motion compensation on the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed, where the prediction block of the image block to be processed includes a prediction value of the target sample; and performing a weighting calculation on the one or more recovered values of the one or more reference samples and the target sample prediction value to update the target sample prediction value, where the reference sample has a given spatial position relationship with the target sample.

Based on this decision, spatial filtering processing is performed on the prediction value of the target sample using the adjacent reconstructed sample to improve compression coding efficiency.

With reference to the first aspect, in a possible implementation, one or more reference samples include a reconstructed sample that has the same horizontal coordinate as the target sample and has a given vertical coordinate difference with the target sample, or a reconstructed sample that has the same vertical coordinate as the target sample and has the given horizontal coordinate difference with the target sample.

Based on this decision, filtering processing is performed on the target sample using a reference sample that has a predetermined spatial position relationship with the target sample. Compared with conventional technology, the coding efficiency is improved.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, являются

,

представляют собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки

и

представляют восстановленные значения опорных выборок на позициях

координат и

, соответственно, w1, w2, w3, w4, w5 и w6 являются заданными константами, и M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

are the prediction value of the target sample before updating,

represents the updated prediction value of the target sample

and

represent the reconstructed values of the reference samples at the positions

coordinates and

, respectively, w1, w2, w3, w4, w5 and w6 are given constants, and M1 and M2 are given positive integers.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementations, in another possible implementation, w1 + w2 = R1, w3 + w4 = R2 or w5 + w6 + w7 = R3, where R1, R2 and R3 are each 2 to the power of n, and n are a non-negative integer.

Based on this decision, coding efficiency can be further improved.

It should be understood that each of R1, R2 and R3 is equal to 2 to the power of n. R1, R2 and R3 are the same or different, and this is not limited. For example, R1, R2, and R3 may be 8, or R1, R2, and R3 may be 2, 4, and 16, respectively.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, являются

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на координатах

,

,

и

позиций, соответственно, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 и w11 являются заданными константами, и M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples on the coordinates

,

,

and

positions, respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 and w11 are given constants, and M1, M2, M3 and M4 are given positive integers.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementations, in another possible implementation, w1 + w2 + w3 = S1, w4 + w5 + w6 = S2 or w7 + w8 + w9 + w10 + w11 = S3, where S1, S2 and S3 are equal 2 to the power of n, and n is a non-negative integer.

Based on this decision, the coding efficiency can be further improved.

It should be understood that each of S1, S2 and S3 is equal to 2 to the power of n. S1, S2 and S3 are the same or different, and this is not limited. For example, S1, S2, and S3 may be 8, or S1, S2, and S3 may be 2, 4, and 16, respectively.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляют собой значение предсказания целевой выборки перед обновлением,

представляют обновленное значение предсказания целевой выборки

и

представляют восстановленные значения опорных выборок на позициях

координат и

, соответственно, w1, w2 и w3 являются заданными константами и M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

are the prediction value of the target sample before updating,

represent the updated prediction value of the target sample

and

represent the reconstructed values of the reference samples at the positions

coordinates and

, respectively, w1, w2 and w3 are given constants and M1 and M2 are given positive integers.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementations, in another possible implementation, w1 + w2 + w3 = R, where R is 2 to the power of n, and n is a non-negative integer.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координат, соответственно, w1, w2, w3, w4 и w5 являются заданными константы и M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1, w2, w3, w4 and w5 are given constants and M1, M2, M3 and M4 are given positive integers.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementations, in another possible implementation, w1 + w2 + w3 + w4 + w5 = S, where S is equal to 2 to the power of n, and n is a non-negative integer.

Based on this decision, the coding efficiency can be further improved.

With reference to the first aspect and the above possible implementation, in another possible implementation, one or more reference samples include one or more of the following samples: a reconstructed sample that has the same horizontal coordinate as the target sample and that is near the top side of the image block to be processed, the reconstructed sample that has the same vertical coordinate as the target sample and that is adjacent to the left side of the image block to be processed, the upper right reconstructed sample of the image block to be processed, the lower left reconstructed sample of the block of the image to be processed, or the top left reconstructed sample of the image block to be processed.

Based on this decision, filtering processing is performed on the target sample using a reference sample that has a predetermined spatial position relationship with the target sample. Compared with conventional technology, the coding efficiency is improved.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2)/2))/(w1 + w2)

where predP1(xP, yP) = (predV(xP, yP) + predH(xP, yP) + nTbW * nTbH) >> (Log2(nTbW) + Log2(nTbH) + 1), predV(xP, yP) = ((nTbH – 1 – yP) * p(xP, –1) + (yP + 1) * p(–1, nTbH)) << Log2(nTbW), predH(xP, yP) = ((nTbW – 1 – xP) * p(–1, yP) + (xP + 1) * p(nTbW, –1)) << Log2(nTbH), target sample coordinates are (xP, yP), image block upper left sample coordinates, to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, –1), p(–1, nTbH), p (–1, yP) and p (nTbW, –1) represent the reconstructed values of the reference samples at positions (xP, –1), (–1, nTbH), (–1, yP) and (nTbW, – 1) coordinates, respectively, w1 and w2 are given constants and nTbW and nTbH represent the width and height of the image block to be processed.

In an exemplary implementation of the first aspect, the target sample prediction value is updated according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predV(xP, yP) + w3 * predH(xP, yP) + ((w1 + w2 + w3)/2))/(w1 + w2 + w3)

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH) + nTbH / 2) >> Log2 (nTbH), predH (xP, yP) = ((nTbW - 1 - xP) * p (-1, yP) + (xP + 1) * p (nTbW, -1) + nTbW / 2) >> Log2 (nTbW), coordinates of the target sample are (xP, yP), the coordinates of the upper left sample of the image block to be processed are (0, 0), predP (xP, yP) represents the prediction value of the target sample before updating, predQ (xP, yP) represents the updated prediction value target sample, p(xP, –1), p(–1, nTbH), p(–1, yP), and p(nTbW, –1) represent the reconstructed values of the reference samples at positions (xP, –1), (– 1, nTbH), (-1, yP) and (nTbW, -1) coordinates, respectively, w1, w2 and w3 are given constants, and nTbW and nTbH represent the width and height of the image block to be processed.

In an exemplary implementation of the first aspect, the target sample prediction value is updated according to the following formula:

predQ(xP, yP) = (((w1 * predP(xP, yP)) << (Log2(nTbW) + Log2(nTbH) + 1)) + w2 * predV(xP, yP) + w3 * predH(xP , yP) + (((w1 + w2 + w3)/2) << (Log2(nTbW) + Log2(nTbH) + 1)))/(((w1 + w2 + w3) << (Log2(nTbW) + Log2(nTbH) + 1)))

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH)) << Log2 (nTbW), predH (xP, yP) = ((nTbW - 1 - xP) * p (–1, yP) + (xP + 1) * p (nTbW, –1)) << Log2 (nTbH), target sample coordinates are (xP, yP) , the coordinates of the upper left sample of the image block to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, - 1), p (–1, nTbH), p (–1, yP), and p (nTbW, –1) represent the reconstructed values of the reference samples at positions (xP, –1), (–1, nTbH), (–1 , yP) and (nTbW, –1) coordinates, respectively, w1 and w2 are given constants, and nTbW and nTbH represent the width and height of the image block to be processed.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2)/2))/(w1 + w2)

where

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координат, соответственно, w1 и w2 являются заданными константами и nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1 and w2 are given constants and nTbW and nTbH represent the width and height of the image block to be processed.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementations, in another possible implementation, the sum of w1 and w2 is 2 raised to the power of n, and n is a non-negative integer.

Based on this decision, the coding efficiency can be further improved.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample, to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

where

,

,

,

,

, координаты целевой выборки равны

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

и

представляют восстановленные значения опорных выборок на позициях

,

и

координат, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, и clip1Cmp представляет операцию отсечения.

, the coordinates of the target sample are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

and

represent the reconstructed values of the reference samples at the positions

,

and

coordinates, respectively, nTbW and nTbH represent the width and height of the image block to be processed, and clip1Cmp represents the clipping operation.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementation, in another possible implementation, updating the prediction value of the target sample includes: performing a weighting calculation based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated the prediction value of the target sample is obtained according to the following formula:

where

,

,

,

,

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, clip1Cmp представляет операцию отсечения.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, nTbW and nTbH represent the width and height of the image block to be processed, clip1Cmp represents the clipping operation.

Based on this decision, an updated prediction value of the target sample can be obtained through the filtering processing.

With reference to the first aspect and the above possible implementations, in another possible implementation, performing a weighting calculation on one or more recovered values of one or more reference samples and a prediction value of the target sample includes: when the recovered reference sample value is not available, determining in a given order of availability samples adjacent to the top and left side of the image block to be processed until a predetermined number of available reference samples is obtained; and performing a weighting calculation on the recovered available reference sample value and the target sample prediction value.

Based on this decision, when a reconstructed reference sample value is not available, the reference samples that are located to the left and above the image block to be processed and whose reconstructed values are available can be found in a predetermined order. Thus, the target sample prediction value can be updated using the recovered values of the available reference samples.

With reference to the first aspect and the aforementioned possible implementations, in another possible implementation, determining in a given order the availability of samples adjacent to the top and left side of the image block to be processed until a given number of available reference samples is obtained includes: obtaining the available reference samples in order from coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1).

Based on this decision, reconstructed values of the available reference samples can be obtained.

With reference to the first aspect and the above possible implementations, in another possible implementation, if there is at least one available reference sample in all the reference samples, if the restored value of the reference sample (xN - 1, yN + nTbH - 1) is not available, then search for an available sample in a given order from coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1). Once an available sample is found, the search ends. If the available sample is (x, y), the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is set equal to the reconstructed sample value (x, y). If the reconstructed reference sample value (x, y) is not available in the set including the reference sample (xN - 1, yN + nTbH - M), the reconstructed reference sample value (x, y) is set to the reconstructed sample value (x, y + 1), where M is greater than or equal to 2 and less than or equal to nTbH + 1. If the reconstructed reference sample value (x, y) is not available in the set including the reference sample (xN + N, yN - 1), the reconstructed reference sample value (x, y) is set to the reconstructed value of the reference sample (x - 1, y), where N is greater than or equal to 0 and less than or equal to nTbW - 1.

Based on this decision, reconstructed values of the available reference samples can be obtained.

With reference to the first aspect and the aforementioned possible implementations, in another possible implementation, if the reconstructed reference sample value (xN - 1, yN + nTbH - M) is not available, the available reference sample can be found in the given order starting from the coordinates (xN - 1 , yN + nTbH - M), where M is greater than or equal to 1 and less than or equal to nTbH + 1. If the available reference sample is B, the restored value of the reference sample (xN - 1, yN + nTbH - M) can be set to the restored value reference sample B. If a reconstructed reference sample value at coordinates (xN + N, yN - 1) is not available, an available reference sample can be found in the given order, starting at coordinates (xN + N, yN - 1), where N is greater than or equal to 0 and less than or equal to nTbW - 1. If the available reference sample is equal to C, the restored value of the reference sample (xN + N, yN - 1) can be set to restore the changed value of the reference sample C.

Based on this decision, reconstructed values of the available reference samples can be obtained.

With reference to the first aspect and the above possible implementations, in another possible implementation, if the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is not available, the available sample is searched in the given order from the coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1). Once an available sample is found, the search ends. If the available sample is (x, y), the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is set equal to the reconstructed sample value (x, y). If the reconstructed reference sample value (xN - 1, yN + nTbH - M) is not available, you can search for the available reference sample in the reverse order of the specified one, starting from coordinates (xN - 1, yN + nTbH - M), where M is greater than 1 and less than or equal to nTbH + 1. If the available reference sample is C, the reconstructed reference sample value (xN - 1, yN + nTbH - M) can be set to the reconstructed reference sample value C. If the reconstructed reference sample value with coordinates (xN + N, yN - 1) is not available, the available reference sample can be found in reverse order, starting at (xN + N, yN - 1) where N is greater than or equal to 0 and less than or equal to nTbW - 1. If the available reference sample is D, the reconstructed reference sample value (xN + N, yN - 1) can be set equal to the reconstructed reference sample value D.

Based on this decision, reconstructed values of the available reference samples can be obtained.

With reference to the first aspect and the above possible implementations, in another possible implementation, if it is determined that all samples adjacent to the top side and left side of the image block to be processed are not available, the restored value of the reference sample is set to 1 << (bitDepth - 1) , where bitDepth represents the bit depth of the sample value of the reference sample.

Based on this decision, the recovered value of the reference sample can be set based on the bit depth when both the recovered value of the reference sample and the recovered value of the new reference sample are not available.

With reference to the first aspect and the above possible implementations, in another possible implementation, before performing a weighting calculation for one or more reconstructed values of one or more reference samples and a prediction value of the target sample, the method includes: when the reference sample is above the image block to be processing, performing a weighting calculation on the recovered reference sample value and the recovered values of the left neighbor sample and the right neighbor sample of the reference sample; when the reference sample is on the left of the image block to be processed, performing a weighting calculation on the restored value of the reference sample and the restored values of the top neighbor sample and the bottom neighbor sample of the reference sample; and updating the recovered reference sample value using the result of the weighting calculation.

Based on this decision, before filtering processing is performed on the target sample, filtering processing is performed on the recovered reference sample value. Thus, the encoding efficiency can be further improved and the prediction residual can be reduced.

With reference to the first aspect and the above possible implementations, in another possible implementation, before performing motion compensation on the image block to be processed based on the motion information, the method further includes: initially updating the motion information using the first predetermined algorithm; and accordingly, performing motion compensation on the image block to be processed based on the motion information includes: performing motion compensation on the image block to be processed based on the initially updated motion information.

Based on this decision, motion information is updated before motion compensation is performed for the current block, and motion compensation is performed based on the updated motion information. In this way, the prediction residual can be reduced.

With reference to the first aspect and the above possible implementations, in another possible implementation, after obtaining a prediction block of an image block to be processed, the method further includes: pre-updating the prediction block using a second predetermined algorithm; and accordingly, performing a weight calculation on one or more recovered values of one or more reference samples and a target sample prediction value includes: performing a weight calculation on one or more recovered values of one or more reference samples and a preliminary updated target sample prediction value.

Based on this decision, the prediction block of the current block is previously updated, and a weighting calculation is performed based on the previously updated prediction value and the restored reference sample value. In this way, the prediction residual can be reduced.

With reference to the first aspect and the above possible implementations, in another possible implementation, after performing a weighting calculation on one or more recovered values of one or more reference samples and a prediction value of the target sample to update the prediction value of the target sample, the method further includes: updating the value predicting the target sample using the second predetermined algorithm.

Based on this decision, a prediction value that is related to the target sample and that has undergone spatial filtering processing can be updated using a predetermined algorithm. In this way, the prediction residual can be reduced.

With reference to the first aspect and the above possible implementations, in another possible implementation, before performing a weighting calculation for one or more recovered values of one or more reference samples and a prediction value of the target sample, the method further includes: parsing the bitstream to obtain a prediction mode corresponding to the block of the image to be processed; and determining that the prediction mode is a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode. It can be understood that the advanced inter-motion vector prediction (inter-AMVP) mode may also be referred to as the inter-motion vector prediction (inter-MVP) mode.

Based on this decision, before the filtering processing, the prediction mode corresponding to the image block to be processed can be determined.

With reference to the first aspect and the above possible implementations, in another possible implementation, before performing a weighting calculation on one or more recovered values of one or more reference samples and a prediction value of the target sample, the method further includes: parsing the bit stream to obtain indication information determining an update of the image block to be processed; and determining that the update determination indication information is used to indicate to update the prediction block of the image block to be processed.

Based on this decision, update definition indication information of the image block to be processed can be obtained by parsing the bitstream, and it is determined that the prediction block of the image block to be processed is to be updated.

With reference to the first aspect and the above possible implementations, in another possible implementation, before performing a weighting calculation for one or more recovered values of one or more reference samples and a target sample prediction value, the method further includes: obtaining predetermined image block update determination indication information to be processed; and determining that the update determination indication information is used to indicate to update the prediction block of the image block to be processed.

Based on this decision, update determination indication information of the image block to be processed can be obtained, and based on the update determination indication information, it is determined that the prediction block of the image block to be processed is to be updated.

According to a second aspect of the embodiments of the present invention, an inter prediction apparatus is provided, including: a parser configured to parse a bitstream to obtain motion information of an image block to be processed; a compensation module, configured to perform motion compensation on the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed, where the prediction block of the image block to be processed includes a prediction value of the target sample; and a calculation module configured to perform weighting calculations on one or more recovered values of one or more reference samples and a prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a predetermined spatial position relationship with the target sample.

With reference to the second aspect and the above possible implementation, in another possible implementation, one or more reference samples include a reconstructed sample that has the same horizontal coordinate as the target sample and has a given vertical coordinate difference with the target sample, or the reconstructed sample , which has the same vertical coordinate as the target sample and has the given horizontal coordinate difference with the target sample.

With reference to the second aspect and the above possible implementation, in another possible implementation, the calculation module is specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, w1, w2, w3, w4, w5 и w6 являются заданными константами и M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, w1, w2, w3, w4, w5 and w6 are given constants and M1 and M2 are given positive integers.

With reference to the second aspect and the above possible implementations, in another possible implementation, w1 + w2 = R1, w3 + w4 = R2 or w5 + w6 + w7 = R3, where R1, R2 and R3 are each 2 to the power of n, and n is a non-negative integer.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координат, соответственно, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 и w11 являются заданными константами и M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 and w11 are given constants and M1, M2, M3 and M4 are given positive integers.

With reference to the second aspect and the above possible implementations, in another possible implementation, w1 + w2 + w3 = S1, w4 + w5 + w6 = S2 or w7 + w8 + w9 + w10 + w11 = S3, where S1, S2 and S3 are equal 2 to the power of n, and n is a non-negative integer.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, w1, w2 и w3 являются заданными константами и M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, w1, w2 and w3 are given constants and M1 and M2 are given positive integers.

With reference to the second aspect and the above possible implementations, in another possible implementation, w1 + w2 + w3 = R, where R is equal to 2 to the power of n, and n is a non-negative integer.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координат, соответственно, w1, w2, w3, w4 и w5 являются заданными константами и M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1, w2, w3, w4 and w5 are given constants and M1, M2, M3 and M4 are given positive integers.

With reference to the second aspect and the above possible implementations, in another possible implementation, w1 + w2 + w3 + w4 + w5 = S, where S is equal to 2 to the power of n, and n is a non-negative integer.

With reference to the second aspect and the above possible implementation, in another possible implementation, one or more reference samples include one or more of the following samples: a reconstructed sample that has the same horizontal coordinate as the target sample and that is near the top side of the image block to be processed, the reconstructed sample, which has the same vertical coordinate as the target sample and which is adjacent to the left side of the image block to be processed, the reconstructed sample adjacent to the upper right side of the image block to be processed, the reconstructed sample from the lower left side of the image block to be processed, or a reconstructed sample of the upper left side of the image block to be processed.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2)

where predP1 (xP, yP) = (predV (xP, yP) + predH (xP, yP) + nTbW * nTbH) >> (Log2 (nTbW) + Log2 (nTbH) + 1), predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH)) << Log2 (nTbW), predH (xP, yP) = ((nTbW - 1 - xP) * p (–1, yP) + (xP + 1) * p (nTbW, –1)) << Log2 (nTbH), target sample coordinates are (xP, yP), upper left sample coordinates of the image block, to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, –1), p(–1, nTbH), p (–1, yP) and p (nTbW, –1) represent the reconstructed values of the reference samples in the coordinates (xP, –1), (–1, nTbH), (–1, yP) and (nTbW, – 1), respectively, w1 and w2 are given constants, and nTbW and nTbH represent the width and height of the image block to be processed.

In an exemplary implementation of the second aspect, the target sample prediction value is updated according to the following formula:

predQ (xP, yP) = (w1 * predP (xP, yP) + w2 * predV (xP, yP) + w3 * predH (xP, yP) + ((w1 + w2 + w3) / 2)) / (w1 + w2 + w3)

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH) + nTbH / 2) >> Log2 (nTbH), predH (xP, yP) = ((nTbW - 1 - xP) * p (-1, yP) + (xP + 1) * p (nTbW, -1) + nTbW / 2) >> Log2 (nTbW), coordinates of the target sample are (xP, yP), the coordinates of the upper left sample of the image block to be processed are (0, 0), predP (xP, yP) represents the prediction value of the target sample before updating, predQ (xP, yP) represents the updated prediction value target sample, p(xP, –1), p(–1, nTbH), p(–1, yP), and p(nTbW, –1) represent the reconstructed values of the reference samples in coordinates (xP, –1), (– 1, nTbH), (-1, yP) and (nTbW, -1), respectively, w1, w2 and w3 are given constants and nTbW and nTbH represent the width and height of the image block to be processed.

In an exemplary implementation of the second aspect, the target sample prediction value is updated according to the following formula:

predQ (xP, yP) = (((w1 * predP (xP, yP)) << (Log2 (nTbW) + Log2 (nTbH) + 1)) + w2 * predV (xP, yP) + w3 * predH (xP , yP) + (((w1 + w2 + w3) / 2) << (Log2 (nTbW) + Log2 (nTbH) + 1))) / (((w1 + w2 + w3) << (Log2 (nTbW) + Log2 (nTbH) + 1)))

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH)) << Log2 (nTbW), predH (xP, yP) = ((nTbW - 1 - xP) * p (–1, yP) + (xP + 1) * p (nTbW, –1)) << Log2 (nTbH), target sample coordinates are (xP, yP) , the coordinates of the upper left sample of the image block to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, - 1), p (–1, nTbH), p (–1, yP), and p (nTbW, –1) represent the reconstructed values of the reference samples at positions (xP, –1), (–1, nTbH), (–1 , yP) and (nTbW, –1) coordinates, respectively, w1 and w2 are given constants, and nTbW and nTbH represent the width and height of the image block to be processed.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2)

where

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координат, соответственно, w1 и w2 являются заданными константами и nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1 and w2 are given constants and nTbW and nTbH represent the width and height of the image block to be processed.

With reference to the second aspect and the above possible implementations, in another possible implementation, the sum of w1 and w2 is equal to 2 raised to the power of n, and n is a non-negative integer.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

where

,

,

,

,

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

и

представляют восстановленные значения опорных выборок на позициях

,

и

координат, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, и clip1Cmp представляет операцию отсечения.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

and

represent the reconstructed values of the reference samples at the positions

,

and

coordinates, respectively, nTbW and nTbH represent the width and height of the image block to be processed, and clip1Cmp represents the clipping operation.

With reference to the second aspect and the above exemplary implementation, in another exemplary implementation, the calculation module is further specifically configured to perform weighting calculations based on the prediction value of the target sample before updating and the restored value of the reference sample to obtain an updated prediction value of the target sample, where the updated prediction value of the target sample The sample is obtained according to the following formula:

where

,

,

,

,

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке и clip1Cmp представляет операцию отсечения.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, nTbW and nTbH represent the width and height of the image block to be processed, and clip1Cmp represents the clipping operation.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is further configured to: when the recovered value of the reference sample is not available, to determine in a given order the availability of samples adjacent to the top and left sides of the image block to be processed, until until a predetermined number of available reference samples are received; and performing a weighting calculation of the reconstructed value of the available reference sample and the prediction value of the target sample.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is specifically configured to obtain the reconstructed value of the available reference sample in order from coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1).

With reference to the second aspect and the above possible implementations, in another possible implementation, when there is at least one available reference sample in all the reference samples, if the restored value of the reference sample (xN - 1, yN + nTbH - 1) is not available, the available the sample is searched in the given order from coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - one). Once an available sample is found, the search ends. If the available sample is (x, y), the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is set equal to the reconstructed sample value (x, y). If the reconstructed reference sample value (x, y) is not available in the set including the reference sample (xN - 1, yN + nTbH - M), the reconstructed reference sample value (x, y) is set to the reconstructed sample value (x, y + 1), where M is greater than or equal to 2 and less than or equal to nTbH + 1. If the reconstructed reference sample value (x, y) is not available in the set including the reference sample (xN + N, yN - 1), the reconstructed reference sample value ( x, y) is set equal to the reconstructed value of the reference sample (x - 1, y), where N is greater than or equal to 0 and less than or equal to nTbW - 1.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is specifically configured to: if the recovered reference sample value (xN - 1, yN + nTbH - M) is not available, the search for an available reference sample in a given order starts from the coordinates ( xN - 1, yN + nTbH - M), where M is greater than or equal to 1 and less than or equal to nTbH + 1. If the available reference sample is equal to B, the reconstructed reference sample value (xN - 1, yN + nTbH - M) can be set to the reconstructed reference sample value B. If the reconstructed reference sample value at coordinates (xN + N, yN - 1) is not available, an available reference sample can be searched in the specified order, starting at coordinates (xN + N, yN - 1), where N is greater than or equal to 0 and less than or equal to nTbW - 1. If the available reference sample is equal to C, the reconstructed reference sample value (xN + N, yN - 1) may be set to the reconstructed reference sample value C.

With reference to the second aspect and the above possible implementations, in another possible implementation, if the reconstructed value of the reference sample (xN - 1, yN + nTbH - 1) is not available, the available sample is searched in the given order from the coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1). Once an available sample is found, the search ends. If the available sample is equal to (x, y), the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is set equal to the reconstructed sample value (x, y). If the reconstructed reference sample value (xN - 1, yN + nTbH - M) is not available, you can search for the available reference sample in the reverse order of the specified one, starting from coordinates (xN - 1, yN + nTbH - M), where M is greater than 1 and less than or equal to nTbH + 1. If the available reference sample is C, the reconstructed reference sample value (xN - 1, yN + nTbH - M) can be set to the reconstructed reference sample value C. If the reconstructed reference sample value with coordinates (xN + N, yN - 1) is not available, the available reference sample can be found in reverse order, starting at (xN + N, yN - 1) where N is greater than or equal to 0 and less than or equal to nTbW - 1. If the available reference sample is D, the reconstructed reference sample value (xN + N, yN - 1) can be set equal to the reconstructed reference sample value D.

With reference to the second aspect and the above possible implementations, in another possible implementation, if it is determined that all samples adjacent to the top side and left side of the image block to be processed are not available, the restored value of the reference sample is set to 1 << (bitDepth - 1 ), where bitDepth represents the bit depth of the sample value of the reference sample.

With reference to the second aspect and the aforementioned possible implementations, in another possible implementation, the calculation module is further configured to: when the reference sample is located over the image block to be processed, to perform a weighting calculation for the restored value of the reference sample and the restored values of the left adjacent sample and the right neighboring sample reference sample; when the reference sample is located to the left of the image block to be processed, perform a weighting calculation for the restored value of the reference sample and the restored values of the upper neighbor sample and the lower neighbor sample of the reference sample; and update the recovered reference sample value using the result of the weighting calculation.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is further configured to initially update the motion information using the first predetermined algorithm; and accordingly, the compensation unit is specifically configured to perform motion compensation of the image block to be processed based on the initially updated motion information.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is further configured to pre-update the prediction block using the second predetermined algorithm; and accordingly, the calculation module is specifically configured to perform weighting calculation of one or more recovered values of one or more reference samples and the previously updated prediction value of the target sample.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is further configured to update target sample prediction values using a second predetermined algorithm.

With reference to the second aspect and the above possible implementations, in another possible implementation, the parsing module is further configured to: parse the bitstream to obtain a prediction mode corresponding to the image block to be processed; and determine that the prediction mode is a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode. It can be understood that the advanced inter-motion vector prediction (inter-AMVP) mode may also be referred to as the inter-motion vector prediction (inter-MVP) mode.

With reference to the second aspect and the aforementioned possible implementations, in another possible implementation, the parsing module is further configured to: perform parsing of the bitstream to obtain update determination indication information of the image block to be processed; and determine that the update determination indication information is used to indicate to update the prediction block of the image block to be processed.

With reference to the second aspect and the above possible implementations, in another possible implementation, the calculation module is further configured to: obtain predetermined update determination information of an image block to be processed; and determine that the update determination indication information is used to indicate to update the prediction block of the image block to be processed.

According to a third aspect of the present invention, a motion information predictor is provided, including a processor and a memory that is coupled to the processor. The processor is configured to perform the method according to the first aspect.

According to a fourth aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores the instruction. When the instruction is executed on the computer, the computer is enabled to execute the method according to the first aspect.

According to a fifth aspect of the present invention, a computer program product including an instruction is provided. When the instruction is executed on the computer, the computer is enabled to execute the method according to the first aspect.

It should be understood that the technical solutions in the second to fifth aspects of the present invention are consistent with those in the first aspect of the present invention. The positive effects achieved in all aspects and the corresponding possible realizations of all aspects are similar. Therefore, details are not described again.

Brief description of the drawings

Fig. 1 is a block diagram of an example of a video coding system that may be configured to be used in an embodiment of the present invention;

fig. 2 is a block diagram of an example of a video encoder that may be configured to be used in an embodiment of the present invention;

fig. 3 is a block diagram of an example of a video decoder that may be configured to be used in an embodiment of the present invention;

fig. 4 is a block diagram of an example of an inter prediction module that can be configured to be used in an embodiment of the present invention;

fig. 5 is a flowchart of an exemplary implementation of the merge prediction mode;

fig. 6 is a flowchart of an exemplary implementation of the advanced motion vector prediction mode;

fig. 7 is a flowchart of an exemplary implementation of motion compensation performed by a video decoder that may be configured for use in an embodiment of the present invention;

fig. 8 is a diagram of an example of an encoding block and an adjacent image block associated with the encoding block;

fig. 9 is a flowchart of an exemplary implementation of listing predicted motion vector candidates;

fig. 10 is a diagram of an exemplary implementation of adding a combined motion vector candidate to a predicted motion vector candidate list in a merge mode;

fig. 11 is a diagram of an exemplary implementation of adding a scaled motion vector candidate to a predicted motion vector candidate list in a merge mode;

fig. 12 is a diagram of an exemplary implementation of adding a zero motion vector to a list of predicted motion vector candidates in a merge mode;

fig. 13 is a flowchart of an inter prediction method according to an embodiment of the present invention;

fig. 14 is an application diagram 1 of an inter prediction method according to an embodiment of the present invention;

fig. 15 is an application diagram 2 of an inter prediction method according to an embodiment of the present invention;

fig. 16 is an application diagram 3 of an inter prediction method according to an embodiment of the present invention;

fig. 17 is an application diagram 4 of an inter prediction method according to an embodiment of the present invention;

fig. 18 is a diagram of an inter prediction device according to an embodiment of the present invention; and

fig. 19 is a diagram of another inter prediction device according to an embodiment of the present invention.

Description of Embodiments

The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention.

Fig. 1 is a block diagram of an example of a video coding system according to an embodiment of the present invention. As described herein, the term "video encoder" generally refers to both a video encoder and a video decoder. In the present invention, the term "video coding" or "coding" can generally refer to video coding or video decoding. Video encoder 100 and video decoder 200 in a video coding system are configured to predict motion information, such as a motion vector, of the current encoded picture block or a sub-block of the current encoded picture block according to various method examples described based on any of the many new inter prediction modes provided herein. of the invention, so that the predicted motion vector closely approximates the motion vector obtained by the motion estimation method, and it is not necessary to transmit the motion vector difference during encoding. In addition, it improves encoding performance.

As shown in FIG. 1, the video coding system includes a source device 10 and a destination device 20. The source device 10 generates encoded video data. Therefore, the source device 10 may be referred to as a video encoding device. The destination device 20 can decode the encoded video data generated by the source device 10 . Therefore, the destination device 20 may be referred to as a video decoding device. In various implementations, source device 10, destination device 20, or both source device 10 and destination device 20 may include one or more processors and memory coupled to one or more processors. Memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store the necessary program code in the form of an instruction or data structure and that can be accessed by a computer as described. in this specification.

Source device 10 and destination device 20 may include various devices including a desktop computer, a mobile computing device, a portable computer (eg, a laptop computer), a tablet computer, a set-top box, a portable telephone. such as a smart phone, TV, camera, display device, digital media player, game console, trip computer, and the like.

Destination device 20 may receive encoded video data from source device 10 via link 30 . Link 30 may include one or more media or devices that can transmit encoded video data from source device 10 to destination device 20. In one example, link 30 may include one or more media that allows source device 10 to directly transmit the encoded video data to the destination device 20 in real time. In this example, source device 10 may modulate the encoded video data in accordance with a communication standard (eg, a wireless communication protocol) and may transmit the modulated video data to destination device 20. One or more communication media may include wireless and/or wired communication media, such as radio frequency (radio frequency, RF) spectrum or one or more physical transmission lines. One or more communication media may form part of a packet data network, and the packet data network is, for example, a local area network, a regional network, or a wide area network (eg, the Internet). One or more communication media may include a router, switch, base station, or other device that provides communication from source device 10 to destination device 20.

In another example, the encoded data may be output to the storage device 40 via the output interface 140 . Similarly, the encoded data can be accessed from the storage device 40 via the input interface 240 . The storage device 40 may include any one of a variety of distributed storage media or locally available storage media, such as a hard disk drive, a Blu-ray disc, a digital video disc (DVD), a compact disc, a read-only memory device (compact disk read-only memory, CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage device capable of storing encoded video data.

In another example, storage device 40 may correspond to a file server or other intermediate storage device configured to store encoded video generated by source device 10. The destination device 20 may access the stored video data from the storage device 40 via streaming or downloading. The file server may be any type of server that can store the encoded video data and transmit the encoded video data to the destination device 20 . In one example, a file server includes a network server (eg, used for a website), a file transfer protocol (FTP), a network-attached storage (NAS) device, or a local drive. The destination device 20 can access the encoded video data via any standard data connection (including an Internet connection). A typical data connection may include a wireless link (for example, a wireless-fidelity (Wi-Fi) connection), a wired connection (for example, a digital subscriber line (DSL) or cable modem), or a combination wireless channel and wired connection, where the combination is suitable for accessing encoded video data stored on a file server. The encoded video data may be transferred from the storage device 40 via streaming, upload-to-download, or a combination thereof.

The motion vector prediction technology of the present invention is applicable to video coding to support a variety of multimedia applications, such as terrestrial TV broadcast, cable TV transmission, satellite TV transmission, video streaming (for example, via the Internet), encoding of video data stored on a storage medium, decoding of video data stored on a storage medium, or another application. In some examples, the video coding system may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcast, and/or video telephony.

The video coding system described in FIG. 1 is just an example, and the technology of the present invention is applicable to video encoding settings (eg, video encoding or video decoding) that do not necessarily include communication between an encoding device and a decoding device. In another example, data is retrieved from local memory, streamed over a network, and the like. The video encoding device may encode the data and store the data in memory, and/or the video decoding device may retrieve the data from the memory and decode the data. In many examples, encoding and decoding are performed by devices that do not communicate with each other and simply encode data and store data in memory and/or retrieve data from memory and decode data.

In the example in FIG. 1, the source device 10 includes a video source 120, a video encoder 100, and an output interface 140. In some examples, output interface 140 may include a modulator/demodulator (modem) and/or a transmitter. Video source 120 may include a video capture device (e.g., a video camera), a video archive including previously captured video data, a video input interface for receiving video data from a video content provider, and/or a computer graphics system for generating video data or combinations of the above video data sources.

Video encoder 100 may encode video data from video source 120. In some examples, source device 10 directly transmits encoded video data to destination device 20 via output interface 140 . In other examples, the encoded video data may be further stored in the storage device 40 so that the destination device 20 subsequently accesses the encoded video data for decoding and/or playback.

In the example in FIG. 1, the destination device 20 includes an input interface 240, a video decoder 200, and a display device 220. In some examples, input interface 240 includes a receiver and/or a modem. Input interface 240 may receive encoded video data over link 30 and/or from storage device 40. The display device 220 may be integrated with the destination device 20 or may be located outside the destination device 20 . Typically, display device 220 displays decoded video data. The display device 220 may include many types of display devices, such as a liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other type of display device.

In some aspects, although not shown in FIG. 1, video encoder 100 and video decoder 200 may be integrated with an audio encoder and an audio decoder, respectively, and may include an appropriate multiplexer/demultiplexer unit or other hardware and software to encode both audio and video in the same data stream or separate data streams. . In some examples, if applicable, the demultiplexer unit (MUX-DEMUX) may conform to the International Telecommunication Union (ITU) H.223 multiplexer protocol or another protocol, such as user datagram protocol (UDP).

Video encoder 100 and video decoder 200 may each be implemented in any of a variety of circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), user-programmable gate matrices (field programmable gate array, FPGA), discrete logic, hardware, or any combination of them. If the present invention is implemented in part using software, a device may store instructions for the software on an appropriate non-volatile computer-readable storage medium and may use one or more processors to execute the instruction in hardware to implement the technology of the present invention. Any of the foregoing (including hardware, software, a combination of hardware and software, and the like) may be considered one or more processors. Video encoder 100 and video decoder 200 may each be used in one or more encoders or decoders. The encoder or decoder may be integrated as part of a combined encoder/decoder (codec) in an appropriate device.

In the present invention, video encoder 100 may generally be referred to as a device that "signals" or "sends" some information to another device, such as video decoder 200. The term "signaling" or "sending" may generally refer to the transmission of a syntax element and/or other data used to decode the compressed video data. The transmission may be in real time or near real time. Alternatively, communication may be performed after a certain period of time has elapsed, such as when a syntax element in the encoded bitstream is stored on a computer-readable storage medium during encoding. The decoding device can then extract the syntax element at any time after the syntax element is stored on the medium.

JCT-VC has developed the H.265 (high efficiency video coding, HEVC) standard. The HEVC standardization is based on an evolved video decoding device model, where the model is referred to as the HEVC test model (HEVC model, HM). The latest standard H.265 document is available at http://www.itu.int/rec/T-REC-H.265. The latest version of the standard document is H.265 (12/16), and the standard document is incorporated herein by reference in its entirety. HM assumes that the video decoder has several additional capabilities over the existing algorithm in ITU-TH.264/AVC. For example, H.264 provides nine intra prediction coding modes, while HM can provide up to 35 intra prediction coding modes.

JVET is developing the H.266 standard. The H.266 standardization process is based on an advanced video decoding device model, where the model is referred to as the H.266 test model. H.266 algorithm descriptions are available at http://phenix.int-evry.fr/jvet and the latest algorithm descriptions are in JVET-F1001-v2. The algorithm description document is incorporated herein by reference in its entirety. In addition, the reference software for the JEM test model is available at https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/ and is also incorporated herein by reference in its entirety.

Typically, as described in the HM Working Model, a video frame or picture may be divided into a sequence of tree blocks or large coding units (LCUs) including both luminance and chrominance samples. An LCU is also called a coding tree unit (CTU). The tree block has a function similar to that of the macroblock in the H.264 standard. A segment includes several consecutive tree blocks in decoding order. A video frame or image may be divided into one or more segments. Each tree block may be divided into coding blocks based on the quadtree. For example, a tree block serving as the root node of a quadtree may be split into four child nodes, and each child node may also serve as a parent node and be split into four other child nodes. The final non-splittable child node serving as a leaf node of the quadtree includes a decoding node, such as a decoded video block. The maximum number of times a tree block may be split and the minimum decoding node size may be specified in the syntax data associated with the decoded bitstream.

The coding unit includes a decoding node, a prediction unit (PU), and a transform unit (TU) associated with the decoding node. The size of the CU corresponds to the size of the decoding node, and the shape of the CU must be a square. The size of a CU can vary from 8×8 pixels to a maximum of 64×64 pixels, or it can be a larger tree block size. Each CU may include one or more PUs and one or more TUs. For example, the syntax data associated with a CU may describe the division of one CU into one or more PUs. The division modes may change when the CU is encoded based on the skip or direct prediction mode, encoded based on the intra prediction mode, or encoded based on the inter prediction mode. The PU resulting from the separation may have a non-square shape. For example, syntax data associated with a CU may alternatively describe the division of one CU into one or more TUs based on a quadtree. The TU may be square or non-square.

The HEVC standard allows conversion based on TU. Tus may be different for different CUs. The TU size is usually set based on the size of the PUs within a given CU defined for a partitioned LCU. However, this is not always the case. The TU size is usually the same or smaller than the PU size. In some possible implementations, a quadtree structure referred to as a "residual quadtree" (residual quadtree, RQT) may be used to divide the residual sample corresponding to a CU into smaller blocks. The leaf node of an RQT may be referred to as a TU. The pixel difference associated with the TU may be transformed to generate a transform coefficient, and the transform coefficient may be quantized.

Typically, the PU includes data related to the prediction process. For example, when the PU is encoded based on the intra prediction mode, the PU may include data describing the intra prediction mode corresponding to the PU. In another possible implementation, when the PU is encoded based on the inter prediction mode, the PU may include data defining a motion vector of the PU. For example, the data defining the motion vector of the PU may describe the horizontal motion vector component, the vertical motion vector component, the resolution (e.g., 1/4 sample precision or 1/8 sample precision) of the motion vector, the reference picture pointed to by the motion vector, and /or a reference picture list (eg, list 0, list 1, or list C) of the motion vector.

Typically, transformation and quantization processes are used for TUs. A given CU including one or more PUs may also include one or more TUs. After prediction, video encoder 100 may calculate a residual value corresponding to the PU. The residual value includes the pixel difference. The pixel difference may be converted to a transform coefficient, and the transform coefficient is quantized and scanned by the TU to generate a serialized transform coefficient for entropy decoding. In the present invention, the term "video unit" is generally used to refer to a decoding unit CU. In some specific applications in the present invention, the term "video block" may also be used to refer to a tree block, such as an LCU or CU, including a decoding node, a PU, and a TU.

A video sequence typically includes a series of video frames or pictures. For example, a group of pictures (GOP) includes a series of video images or one or more video images. The GOP may include the syntax data in the header information of the GOP, in the header information of one or more pictures, or elsewhere, and the syntax data describes the number of pictures included in the GOP. Each image segment may include segment syntax data describing an encoding mode for the corresponding image. Video encoder 100 typically performs an operation on a video block in a video segment to encode video data. The video block may correspond to a decoding node in the CU. The video block size may be fixed or variable and may vary depending on the specified decoding standard.

In an exemplary implementation, the HM supports multi-size PU prediction. Assuming that the size of a given CU is 2N × 2N, HM supports intra prediction for a PU with a size of 2N × 2N or N × N, and an inter prediction for a symmetric PU with a size of 2N × 2N, 2N × N, N × 2N or N × N The HM also supports asymmetric partitioning for inter-prediction for PU sizes of 2N × nU, 2N × nD, nL × 2N, or nR × 2N. In asymmetric splitting, the CU is not split in one direction, but is split into two parts in the other direction, where one part is 25% CU and the other part is 75% CU. The portion that accounts for 25% of CU is indicated by an indicator including "n" followed by "U (up)", "D (down)", "L (left)", or "R (right)". Therefore, for example, "2N × nU" refers to a horizontally divided 2N × 2N CU with a 2N × 0.5N PU at the top and a 2Nx1.5N PU at the bottom.

In the present invention, "N×N" and "N times N" are used interchangeably to indicate the pixel size of a video block in vertical dimension and horizontal dimension, such as 16×16 pixels or 16 times 16 pixels. Typically a 16×16 block has 16 pixels in the vertical direction (y = 16) and 16 pixels in the horizontal direction (x = 16). Similarly, an N × N block has N pixels in the vertical direction and N pixels in the horizontal direction, where N is a non-negative integer. Pixels in a block can be arranged in rows and columns. In addition, in a block, the number of pixels in the horizontal direction and the number of pixels in the vertical direction may not necessarily be the same. For example, a block may include N × M pixels, where M is not necessarily equal to N.

After intra- or inter-prediction decoding has been performed for the PU in the CU, video encoder 100 may calculate the residual data of the TU in the CU. The PU may include pixel data in a spatial domain (also referred to as a pixel domain). The TU may include a coefficient in the transform domain after a transform (eg, a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform) is applied to the residual video data. The residual data may correspond to a pixel difference between the pixels of an uncoded image and a prediction value corresponding to the PU. Video encoder 100 may generate a TU including the CU residual data and then transform the TU to generate a transform coefficient of the CU.

After performing any transformation to generate the transform coefficients, video encoder 100 may quantize the transform coefficients. Quantization refers, for example, to the process of quantizing the coefficients to reduce the amount of data used to represent the coefficients and to realize additional compression. The quantization process may reduce the bit depth associated with some or all of the coefficients. For example, during quantization, an n-bit value may be reduced to an m-bit value by rounding where n is greater than m.

The JEM model further improves the video coding structure. In particular, a block coding structure called a Quadtree Plus Binary Tree (QTBT) structure is presented. Without the use of concepts such as CU, PU, and TU in HEVC, the QTBT framework supports a more flexible form of CU partitioning. The CU may be square or rectangular. Quadtree splitting is first performed on the CTU, and then binary tree splitting is performed on the leaf node of the quadtree. In addition, there are two partitioning modes in binary tree partitioning: symmetrical horizontal splitting and symmetrical vertical splitting. The leaf node of a binary tree is called CU. The CU in the JEM model cannot be further split during prediction and transformation. In other words, CU, PU, and TU in the JEM model have the same block size. In the current JEM model, the maximum CTU size is 256 × 256 luma pixels.

In some possible implementations, video encoder 100 may scan the quantized transform coefficient in a predetermined scan order to generate a serialized vector that may be entropy encoded. In other possible implementations, video encoder 100 may perform adaptive scanning. After scanning the quantized transform coefficient to generate a one-dimensional vector, video encoder 100 may perform entropy encoding of the one-dimensional vector using context-based adaptive variable-length code (CAVLC), context-based adaptive binary arithmetic coding (context-based adaptive binary arithmetic coding, CABAC), syntax-based adaptive binary arithmetic coding based on syntax (syntax-based adaptive binary arithmetic coding, SBAC), probability interval partitioning entropy (PIPE), or another entropy coding method. Video encoder 100 may further perform entropy encoding of the syntax element associated with the encoded video data so that video decoder 200 decodes the video data.

To perform CABAC, video encoder 100 may assign a context in a context model to a character to be transmitted. The context may be related to whether the character's adjacent value is non-null. To perform CAVLC, video encoder 100 may select a variable length code for the transmitted symbol. A codeword may be generated by variable-length coding (VLC) such that a shorter code corresponds to a more likely symbol and a longer code corresponds to a less likely symbol. Thus, compared to using codewords of the same length for all symbols to be transmitted, the use of VLC can reduce the bit rate. The probability in CABAC can be determined based on the context assigned to the symbol.

In this embodiment of the present invention, to reduce temporal redundancy between pictures, the video encoder may perform inter-prediction. As described above, a CU may have one or more PUs depending on different video compression coding standards. In other words, multiple PUs may belong to the same CU, or PUs and CUs may be the same size. In this specification, when the CU and the PU are the same size, the division mode corresponding to the CU is that the CU is not divided, or the CU is divided into one PU, and the PU is uniformly used for description. When the video encoder performs inter prediction, the video encoder may send the motion information of the PU to the video decoder. For example, the PU motion information may include a reference picture index, a motion vector, and a prediction direction indicator. The motion vector may indicate an offset between an image block (also called a video block, pixel block, pixel set, or the like) PU and a reference PU. The reference PU may be part of a reference picture, similar to the picture PU. The reference block may be positioned as a reference picture indicated by a reference picture index and a prediction direction indicator.

To reduce the number of coding bits required to represent PU motion information, the video encoder may generate a list of predicted motion vectors (Motion Vector, MV) for each PU based on the merge prediction mode or the advanced motion vector prediction mode. Each predicted motion vector candidate in the predicted motion vector candidate list for the PU may indicate motion information. The motion information indicated by some of the predicted motion vector candidates in the predicted motion vector candidate list may be based on the motion information of other PUs. If the predicted motion vector candidate indicates motion information of one of the specified spatial candidate, the predicted motion vector at a position, or the specified temporal predicted motion vector candidate at a position, the predicted motion vector candidate may be referred to as the "original" predicted motion vector candidate in the present invention. For example, in the merge mode, also referred to as the merge prediction mode in this specification, there may be five initial spatial position predicted motion vector candidates and one initial position predicted motion vector candidate temporal. In some examples, the video encoder may generate additional predicted motion vector candidates by combining some motion vectors from different original predicted motion vector candidates, modifying the original predicted motion vector candidate, or inserting only the null motion vector as a predicted motion vector candidate. The predicted motion vectors of the additional candidates are not considered to be the predicted original motion vector candidates and may be referred to as artificially generated predicted motion vector candidates in the present invention.

The techniques in the present invention typically include a technique for listing predicted motion vector candidates at a video encoder and a technique for listing the same predicted motion vector candidates at a video decoder. The video encoder and the video decoder may generate the same motion vector predicted candidate list by implementing the same technology for generating the motion vector predicted candidate list. For example, the video encoder and video decoder may list the same number of motion vector predictor candidates (eg, five motion vector predictor candidates). The video encoder and video decoder may first consider the predicted motion vectors of spatial candidates (eg, neighboring blocks in the same picture) and then consider the predicted motion vectors of temporal candidates (eg, predicted motion vector candidates in different pictures), and finally may consider artificially generated predicted vector candidates. until the required number of predicted motion vector candidates are added to the lists. According to the techniques in the present invention, during the listing of motion vector predicted candidates, a pruning operation may be performed on some types of predicted motion vector candidates to remove duplicate predicted motion vector candidates from the list of predicted motion vector candidates, but may not be performed on other types of predicted motion vector candidates. to reduce the complexity of the decoder. For example, for a set of predicted spatial motion vector candidates and a predicted temporal motion vector candidate, a pruning operation may be performed to remove a predicted motion vector candidate with the same motion information from the list of predicted motion vector candidates. However, the artificially generated motion vector predictor candidate may be added to the motion vector predictor candidate list without being removed.

After generating a list of predicted motion vector candidates for the PU from the CU, the video encoder may select a predicted motion vector candidate from the list of predicted motion vector candidates and output an index of the predicted motion vector candidate in the bitstream. The selected predicted motion vector candidate may be a predicted motion vector candidate for generating a motion vector that most closely matches the prediction value of the target PU that is being decoded. The motion vector predictor candidate index may indicate the position of the selected motion vector candidate in the predicted motion vector candidate list. The video encoder may further generate a PU prediction image block based on the reference block indicated by the PU motion information. The motion information of the PU may be determined based on the motion information indicated by the selected motion vector predictor candidate. For example, in the merge mode, the motion information of the PU may be the same as the motion information indicated by the selected motion vector predictor candidate. In the AMVP mode, the motion information of the PU may be determined based on the difference of the motion vectors for the PU and the motion information indicated by the selected motion vector prediction candidate. The video encoder may generate one or more CUs based on the CU prediction PU and the original CU. The video encoder may then encode one or more afterimage blocks and output one or more afterimage blocks in the bitstream.

The bitstream may include data identifying a selected predicted motion vector candidate in the list of predicted motion vector candidates for the PU. The video decoder may determine the motion information of the PU based on the motion information indicated by the selected motion vector predictor candidate in the predicted motion vector candidate list for the PU. The video decoder may identify one or more PU reference blocks based on the motion information of the PU. After identifying one or more PUs, the video decoder may generate a PU prediction picture block based on the one or more PUs. The video decoder may reconstruct the CU image block based on the CU prediction image block PU and one or more residual CU image blocks.

For ease of explanation, in the present invention, a position or image block may be described as a position or image block having a different spatial relationship with a CU or PU. The description can be explained as follows: a position or image block has a different spatial relationship with an image block associated with a CU or PU. Further, in the present invention, the PU currently being decoded by the video decoder may be referred to as the current PU and also referred to as the current image block to be processed. In the present invention, the CU currently being decoded by the video decoder may be referred to as the current CU. In the present invention, the picture currently being decoded by the video decoder may be referred to as the current picture. It should be understood that the present invention is also applicable to the case where the PU and CU have the same size or the PU is a CU. PU is used uniformly for description.

As briefly described above, video encoder 100 may generate a prediction picture block and CU PU motion information by inter prediction. In many examples, the motion information of a given PU may be the same or similar to the motion information of one or more neighboring PUs (namely, a PU whose image block is spatially or temporally adjacent to that PU's image block). Because a neighbor PU often has similar motion information, video encoder 100 may encode the motion information of a given PU based on the neighbor PU's motion information. Encoding the motion information of a given PU based on the motion information of a neighbor PU can reduce the number of coding bits in a bitstream that are required to indicate the motion information of a given PU.

Video encoder 100 may encode the motion information of a given PU based on the motion information of a neighboring PU in various ways. For example, video encoder 100 may indicate that the motion information of a given PU is the same as the motion information of a neighboring PU. In the present invention, the merge mode may be used to indicate that the motion information of a given PU is the same as or can be obtained from the motion information of a neighboring PU. In another possible implementation, video encoder 100 may compute a Motion Vector Difference (MVD) for a given PU. MVD indicates the difference between the motion vector of a given PU and that of a neighbor PU. Video encoder 100 may add an MVD instead of a given PU's motion vector to the given PU's motion information. In a bitstream, the number of coding bits needed to represent an MVD is less than the number of coding bits needed to represent a given PU's motion vector. In the present invention, an advanced motion vector prediction mode can be used to indicate that the motion information of a given PU is transmitted to the decoder side using the MVD and an index value that is used to identify a motion vector candidate.

To signal, based on the merge mode or AMVP mode, the motion information of a given PU to the decoder side, video encoder 100 may generate a list of predicted motion vector candidates for the given PU. The list of predicted motion vector candidates may include one or more predicted motion vector candidates. Each of the motion vector predictor candidates in the motion vector predictor candidate list for a given PU may indicate motion information. The motion information indicated by each predicted motion vector candidate may include a motion vector, a reference picture index, and a prediction direction indicator. The motion vector predictor candidates in the motion vector predictor candidate list may include the "original" motion vector predictor candidate. Each of the motion vector predictor candidates indicates motion information in one of said motion vector predictor candidates at positions within the PU other than the PU.

After generating the list of predicted motion vector candidates for the PU, video encoder 100 may select one predicted motion vector candidate from the list of predicted motion vector candidates that is used for the PU. For example, the video encoder may compare each predicted motion vector candidate with a PU that is being decoded and may select a predicted motion vector candidate with a desired rate distortion cost. Video encoder 100 may output a predicted motion vector candidate index for the PU. The motion vector predictor candidate index may identify the position of the selected motion vector candidate in the predicted motion vector candidate list.

Additionally, video encoder 100 may generate a PU prediction image block based on a reference block indicated by the PU motion information. The motion information of the PU may be determined based on the motion information indicated by the selected motion vector predictor candidate in the PU predicted motion vector candidate list. For example, in the merge mode, the motion information of the PU may be the same as the motion information indicated by the selected motion vector predictor candidate. In the AMVP mode, motion information of the PU may be determined based on the difference of the motion vectors for the PU and the motion information indicated by the selected motion vector predictor candidate. As described above, video encoder 100 may process a PU prediction image block.

When video decoder 200 receives a bit stream, video decoder 200 may generate a list of predicted motion vector candidates for each PU of the CU. The list of predicted motion vector candidates generated by video decoder 200 for the PU may be the same as the list of predicted motion vector candidates generated by video encoder 100 for the PU. The syntax element obtained by parsing the bitstream may indicate the position of the selected motion vector candidate in the list of predicted motion vector candidates for the PU. After generating a candidate list of predicted motion vectors for the PU, video decoder 200 may generate a PU prediction picture block based on one or more reference blocks indicated by the PU's motion information. The video decoder 200 may determine the motion information of the PU based on the motion information indicated by the selected motion vector predictor candidate in the PU predicted motion vector candidate list. Video decoder 200 may reconstruct the CU image block based on the prediction image block PU and the residual CU image block.

It should be understood that, in an exemplary implementation at the decoder side, the listing of motion vector predicted candidates and parsing the bitstream to obtain the position of the selected motion vector candidate in the predicted motion vector candidate list are independent of each other, and may be performed in any order or in parallel.

In another exemplary implementation, the decoder side first obtains the position of the selected motion vector predictor candidate in the predicted motion vector list by parsing the bitstream, and then constructs the list of predicted motion vector candidates based on the position obtained by parsing. In this implementation, it is not necessary to compile all lists of predicted motion vector candidates, and only need to compile a list of predicted motion vector candidates at the position obtained by parsing, that is, provided that the predicted motion vector candidate at the position can be determined. For example, when it is obtained, by bitstream parsing, that the selected motion vector candidate is the motion vector candidate whose index is 3 in the motion vector predictor candidate list, it is only necessary to compile a motion vector predictor candidate list including the predicted vector candidate motion vector candidate whose index is 0 to a predicted motion vector candidate whose index is 3, and a predicted motion vector candidate whose index is 3 can be determined. This can reduce complexity and improve decoding efficiency.

Fig. 2 is a block diagram of an example of a video encoder 100 according to an embodiment of the present invention. Video encoder 100 is configured to output video to post-processing object 41. The post-processing object 41 is an example of a video object that can process the encoded video data from the video encoder 100. For example, the video object is a media-enabled network element (MANE) or a video editing/editing device. In some cases, the post-processing object 41 may be an example of a network object. In some video coding systems, post-processing object 41 and video encoder 100 may be components of separate devices. In other instances, the functions of post-processing object 41 may be performed by the same device, including video encoder 100. In one example, post-processing object 41 is an example of storage device 40 in FIG. one.

In the example in FIG. 2, video encoder 100 includes a prediction processing unit 108, a filter unit 106, a decoded picture buffer (DPB) 107 107, an adder 112, a transformer 101, a quantizer 102, and an entropy encoder 103. The prediction processing unit 108 includes a 110 inter prediction and block 109 intra prediction. To reconstruct the image block, video encoder 100 further includes an inverse quantizer 104, an inverse transform 105, and an adder 111. Filter block 106 is configured to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and sample adaptive offset (SAO) filter. Although the filter block 106 is shown in FIG. 2, in another implementation, filter block 106 may be implemented as a post-loop filter. In one example, video encoder 100 may further include a video memory and a splitter (which is not shown in the drawing).

The video data memory may store video data to be encoded by the video encoder component 100. The video data stored in the video data memory may be obtained from the video source 120. The DPB 107 may be a reference picture memory that stores reference video data used by video encoder 100 to encode video data in an intra coding mode or an inter coding mode. The video memory and DPB 107 may include any of a variety of types of memory devices, such as dynamic random access memory (DRAM) including synchronous dynamic random access memory (SDRAM) , magnetic RAM (magnetic random access memory, MRAM), resistive RAM (resistive random access memory, RRAM), or other type of storage device. The video data memory and DPB 107 may be provided by the same memory device or by separate memory devices. In various examples, the video data memory may be integrated on the chip along with other components of the video encoder 100, or may be located off the chip relative to these components.

As shown in FIG. 2, video encoder 100 receives video data and stores the video data in a video data memory. The splitter divides the video data into multiple image blocks and may further divide these image blocks into smaller blocks, for example, divide these image blocks based on a quadtree structure or a binary tree structure. The division may further include division into segments (slice), tiles (tile) or other larger blocks. Video encoder 100 is typically a component that encodes a picture block in a video segment to be encoded. A segment may be divided into a plurality of image blocks (and may be divided into a set of image blocks called a tile). The prediction processing unit 108 may select one of a plurality of possible coding modes used for the current image block, such as selecting one of a plurality of intra coding modes or one of a plurality of inter coding modes. The prediction processing unit 108 may provide the received intra-coded or inter-coded block to adder 112 for generating a residual block, and provide the received intra-coded or inter-coded block to adder 111 for recovering an encoded block that is used as a reference picture.

Intra predictor 109 in prediction processing block 108 may perform intra prediction encoding of the current block to be encoded with respect to one or more adjacent blocks that are in the same frame or segment as the current image block to remove spatial redundancy. Inter prediction block 110 in prediction processing block 108 may perform inter prediction encoding for the current picture block with respect to one or more prediction blocks in one or more reference pictures to remove temporal redundancy.

In particular, the inter predictor 110 may be configured to determine the inter prediction mode used to encode the current image block. For example, inter predictor 110 may calculate rate distortion values for various inter prediction modes in a suitable inter prediction mode established by rate distortion analysis, and select an inter prediction mode with an optimal rate-distortion ratio function from a set of inter prediction mode candidates. Rate-distortion analysis is generally used to determine the amount of distortion (or error) between the encoded block and the original unencoded block from which the encoded block is generated by encoding, and the data rate (namely, the number of bits) used to generate the encoded block. For example, outer predictor 110 may determine that the inter prediction mode in the inter prediction mode candidate set used to encode the current image block with the minimum data rate distortion cost is the inter prediction mode used to perform inter prediction for the current image block.

The outer predictor 110 is configured to: predict motion information (eg, a motion vector) of one or more sub-blocks of the current image block based on the determined inter prediction mode, and obtain or generate a prediction block of the current image block based on the motion information (eg, motion vector) of one or more several sub-blocks of the current image block. The outer predictor 110 may find in one of the reference picture lists the prediction block pointed to by the motion vector. Outer predictor 110 may further generate a syntax element associated with the image block and video segment such that video decoder 200 uses the syntax element to decode the image block in the video segment. Alternatively, in the example, the outer predictor 110 performs a motion compensation process based on the motion information of each sub-block to generate a prediction block of each sub-block and obtain a prediction block of the current image block. It should be understood that the external predictor 110 in the present description performs a motion estimation process and a motion compensation process.

Specifically, after selecting an inter prediction mode for the current image block, the inter predictor 110 may provide entropy encoder 103 with information indicative of the inter prediction mode selected for the current image block, so that entropy encoder 103 encodes information indicating the selected inter prediction mode.

Intra predictor 109 may perform intra prediction for the current image block. It is understood that the intra predictor 109 may determine the intra prediction mode used to encode the current block. For example, intra predictor 109 may calculate rate distortion values for various intra prediction modes to be tested by rate-distortion analysis, and select an intra prediction mode with an optimal rate-distortion function from among the modes to be tested. In any case, after selecting an intra prediction mode for an image block, intra predictor 109 may provide entropy encoder 103 with information indicative of the intra prediction mode selected for the current image block, so that entropy encoder 103 encodes information indicative of the selected intra prediction mode.

After prediction processing unit 108 generates a prediction block of the current picture block by inter prediction and intra prediction, video encoder 100 obtains a residual picture block by subtracting the prediction block from the current picture block to be encoded. The adder 112 represents one or more components that perform the subtraction operation. The residual video data in the residual block may be included in one or more TUs and provided to a transformer 101. The transformer 101 transforms the residual video data into a residual transform coefficient through a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. The converter 101 may convert the residual video data from the pixel range to a transformation region, such as a frequency domain.

Transformer 101 may send the obtained transform coefficient to quantizer 102. Quantizer 102 quantizes the transform coefficient to further reduce the bit rate. In some examples, quantizer 102 may further scan a matrix including the quantized transform coefficient. Alternatively, entropy encoder 103 may perform a scan.

After quantization, the entropy encoder 103 performs entropy encoding of the quantized transform coefficient. For example, entropy encoder 103 can perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), entropy with probabilistic separation interval (PIPE) coding or other entropy encoding method or technology. After performing entropy encoding, entropy encoder 103 may transmit the encoded bitstream to video decoder 200 or archive the encoded bitstream for later transmission or for later retrieval by video decoder 200. Entropy encoder 103 may further entropy encode the syntax element of the current picture block to be encoded.

An inverse quantizer 104 and an inverse transform 105 respectively perform inverse quantization and inverse transformation to reconstruct a residual block in a pixel region, for example, which is subsequently used as a reference block of a reference image. The adder 111 adds the reconstructed residual block to the prediction block generated by the outer predictor 110 or the inner predictor 109 to generate a reconstructed image block. Filter block 106 is applied to the reconstructed image block to reduce distortion such as block artifacts. Then, the reconstructed image block is used as a reference block and stored in the decoded image buffer 107, and may be used by the outer predictor 110 as a reference block to perform inter block prediction in a subsequent video frame or picture.

It should be understood that other structural variations of video encoder 100 may be used to encode the video stream. For example, for some image blocks or image frames, video encoder 100 may directly quantize the residual signal, processing by transducer 101 is not required and, accordingly, processing by inverse transform 105 is also not required. Alternatively, for some image blocks or image frames, video encoder 100 does not generate residual data and, accordingly, processing by transform 101, quantizer 102, inverse quantizer 104, and inverse transform 105 is not required. Alternatively, video encoder 100 may store the directly reconstructed image block as a reference block, and no processing by filter block 106 is required. Alternatively, quantizer 102 and inverse quantizer 104 in video encoder 100 may be combined.

Fig. 3 is a block diagram of an example of a video decoder 200 according to an embodiment of the present invention. In the example in FIG. 3, video decoder 200 includes an entropy decoder 203, a prediction processor 208, an inverse quantizer 204, an inverse transform 205, an adder 211, a filter block 206, and a DPB 207. The prediction processor 208 may include an outer predictor 210 and an inner predictor 209. In some examples, video decoder 200 may perform a decoding process that is roughly the reverse of the encoding process described with reference to video encoder 100 in FIG. 2.

During decoding, video decoder 200 receives from video encoder 100 an encoded video bitstream that represents a picture block in an encoded video segment and its associated syntax element. The video decoder 200 may receive video data from the network entity 42 and may optionally store the video data in a video data memory (which is not shown in the drawing) in addition. The video data store may store video data, such as an encoded bitstream, to be decoded by the video decoder component 200. The video data stored in the video data store may be obtained, for example, from a local video source, such as a storage device 40 or a camera, via a wired or wireless network transmission of video data or by accessing a physical storage medium. The video data memory may be used as a decoded picture buffer (DPB) for storing encoded video data from an encoded video bitstream. Therefore, although the video data memory is not shown in FIG. 3, the video data memory and the DPB 207 may be the same memory or may be a memory that is located separately. The video memory and DPB 207 may each include any of a variety of types of memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetic RAM (MRAM), resistive RAM (RRAM), or other storage device type. In various examples, the video data memory may be integrated on-chip with other components of video decoder 200, or may be located off-chip relative to these components.

The network entity 42 may be, for example, a server, a MANE, a video editor/editor, or other device capable of implementing one or more of the technologies described above. Network entity 42 may or may not include a video encoder, such as video encoder 100. Before network entity 42 sends the encoded video bitstream to video decoder 200, network entity 42 may implement some of the technologies described in the present invention. In some video decoding systems, network entity 42 and video decoder 200 may be components of separate devices. In other cases, the functions of network entity 42 may be performed by the same device, including video decoder 200. In some instances, network entity 42 may be an example of storage device 40 in FIG. one.

Entropy decoder 203 in video decoder 200 performs entropy decoding of the bitstream to generate a quantized coefficient and some syntax elements. The entropy decoder 203 forwards the syntax elements to the prediction processing block 208 . Video decoder 200 may receive syntax elements at the video segment level and/or the picture block level.

When a video segment is decoded into an intra-decoded (I) segment, intra predictor 209 in prediction processing block 208 may generate a picture block prediction block in the current video segment based on the signaled intra prediction mode and the previously decoded block data of the current frame or picture. When a video segment is decoded into an inter-decoding segment (namely, B or P), the outer predictor 210 in the prediction processing block 208 may determine, based on the syntax elements obtained from the entropy decoder 203, the inter prediction mode to be used for decoding the current picture block in the current segment of the video and decoding (eg, performing inter prediction) of the current image block based on the determined inter prediction mode. In particular, the outer predictor 210 may determine whether a new inter prediction mode is used to predict the current picture block in the current video segment to predict the current picture block. If the syntax element indicates that a new inter prediction mode is used, motion information of the current picture block in the current video segment or motion information of a sub-block of the current picture block is predicted based on the new inter prediction prediction mode (for example, the new inter prediction mode indicated by the syntax element or new default inter prediction mode), and a prediction block of the current image block or a subblock of the current image block is obtained, or generated in a motion compensation process using the predicted motion information that is related to the current image block or a subblock of the current image block. The motion information herein may include image information and a reference motion vector. The reference picture information may include, but is not limited to, single/double prediction information, a reference picture list number, and a reference picture index corresponding to the reference picture list. For inter prediction, a prediction block may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 200 may construct reference picture lists, i.e., list 0 and list 1, based on the reference pictures stored in DPB 207. The current picture's reference frame index may be included in one or both of reference frame list 0 and reference frame list 1. In some examples, video encoder 100 may signal whether a new inter prediction mode is being used to decode a particular syntax element of a particular block, or signaling whether a new inter prediction mode is being used and which new inter prediction mode is being used to decode a particular syntax element of a particular block. It should be understood that here, the outer predictor 210 performs a motion compensation process.

The inverse quantizer 204 performs inverse quantization, that is, dequantizes the quantized transform coefficient provided in the bitstream and decoded by the entropy decoder 203. The inverse quantization process may include: determining the amount of quantization to be applied using a quantization parameter calculated by video encoder 100 for each image block in the video segment, and determining the degree of inverse quantization to be applied in the same way. The detransformer 205 performs an inverse transform, such as a DCT inverse, an integer inverse transform, or a conceptually similar process of inverting a transform coefficient to generate a residual block of a pixel area.

After outer predictor 210 generates a prediction block for the current picture block or a sub-block of the current picture block, video decoder 200 adds a residual block from inverse transform 205 and corresponding to the prediction block generated by outer predictor 210 to obtain a reconstructed block, namely a decoded picture block. . The adder 211 represents a component that performs the addition operation. If necessary, a loop filter (either in the decoding cycle or after the decoding cycle) can be additionally used to smooth out pixel transitions or improve the video quality in another way. Filter block 206 may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and an adaptive sample offset (SAO) filter. Although the filter block 206 is shown in FIG. 2, in another implementation, filter block 206 may be implemented as a post-loop filter. In the example, a filter block 206 is applied to the reconstructed block to reduce block distortion, and the result is output as a decoded video stream. In addition, the decoded image block in a given frame or images may be further stored in the DPB 207, and the DPB 207 stores a reference image used for subsequent motion compensation. The DPB 207 may be part of a memory and may further store decoded video for later presentation to a display device (eg, display device 220 in FIG. 1). Alternatively, the DPB 207 may be separate from such a memory.

It should be understood that other design options for video decoder 200 may be used to decode the encoded video bitstream. For example, video decoder 200 may generate an output video stream and processing by filter block 206 is not required. Alternatively, for some image blocks or image frames, the entropy decoder 203 in the video decoder 200 does not obtain the quantized coefficient through decoding and, accordingly, processing by the inverse quantizer 204 and inverse transform 205 is not required.

As described above, the techniques in the present invention relate to, for example, external decoding. It should be understood that the techniques in the present invention may be performed by any one video codec described in the present invention, and the video decoder includes (for example) video encoder 100 and video decoder 200 shown and described in FIG. 1-fig. 3. To be specific, in a possible implementation, the outer predictor 110 described in FIG. 2 may perform the specific technique described below when inter-prediction is performed during encoding of a video block. In another possible implementation, the outer predictor 210 described in FIG. 3 can perform the specific technique described below when inter-prediction is performed during decoding of a video block. Therefore, reference to a generic "video encoder" or "video decoder" may include video encoder 100, video decoder 200, or other video coding unit or decoding unit.

It should be understood that in the encoder 100 and decoder 200 in the present invention, the processing result of a step may be further processed and then output to the next step. For example, after a step such as interpolation filtering, motion vector output, or contour filtering of the processing result of the corresponding step, an additional operation such as clipping or shifting is performed.

For example, the value of the motion vector is limited to a certain range of bit depth. Assuming the allowed bit depth of the motion vector is bitDepth, the value of the motion vector is in the range -2^(bitDepth - 1) to 2^(bitDepth - 1) - 1, where the character "^" represents exponentiation. If bitDepth is 16, the value is in the range -32768 to 32767. If bitDepth is 18, the value is in the range -131072 to 131071. The motion vector value can be limited in one of the following two ways:

Method 1: Remove the most significant bit of the motion vector overflow:

For example, the value of vx is -32769 and 32767 is output according to the formulas above. The value is stored on the computer in two's complement representation, the two's complement representation of –32769 is 1.0111,1111,1111,1111 (17 bits), and the computer's overflow processing discards the most significant bit. Therefore, the value of vx is 0111,1111,1111,1111, that is, 32767. This value is consistent with the result obtained by processing according to the formulas.

Method 2: Motion vector clipping is performed and the following formulas are used:

In the formulas above, Clip3 is defined as truncation of the z value to the range [x, y]

Fig. 4 is a block diagram of an inter prediction unit 121 according to an embodiment of the present invention. For example, inter prediction module 121 may include a motion estimation block and a motion compensation block. The relationship between PU and CU varies depending on the video compression coding standard. The inter prediction unit 121 may divide the current CU into PUs based on a plurality of division modes. For example, inter prediction unit 121 may divide the current CU into PUs based on the 2N×2N, 2N×N, N×2N, and N×N partition modes. In another embodiment, the current CU is the current PU. It's not limited.

The inter prediction module 121 may perform an integer motion estimation (Integer Motion Estimation, IME) and then a fractional motion estimation (Fraction Motion Estimation, FME) for each PU. When inter prediction module 121 performs an IME for a PU, inter prediction module 121 may search one or more reference pictures for a reference block of the PU. After finding the reference PU, inter prediction module 121 may generate a motion vector that indicates, up to an integer, the spatial offset between the PU and the reference PU. When the inter prediction unit 121 performs the FME for the PU, the inter prediction unit 121 may improve the motion vector generated by performing the IME for the PU. The motion vector generated by performing FME on the PU may have sub-integer precision (eg, 1/2 sample precision or 1/4 sample precision). After generating the motion vector for the PU, inter prediction unit 121 may generate a PU prediction image block using the motion vector of the PU.

In some possible implementations, in which inter prediction module 121 transmits motion information of the PU to the decoder side based on the AMVP mode, inter prediction module 121 may generate a list of predicted motion vector candidates for the PU. The list of motion vector predictor candidates may include one or more original motion vector predictor candidates and one or more additional motion vector predictor candidates derived from the one or more original motion vector predictor candidates. After generating the list of predicted motion vector candidates for the PU, inter prediction module 121 may select a predicted motion vector candidate from the list of predicted motion vector candidates and generate a motion vector difference (MVD) for the PU. The MVD for the PU may indicate the difference between the motion vector indicated by the selected motion vector predictor candidate and the motion vector generated for the PU via the IME and FME. In these possible implementations, inter prediction module 121 may output a predicted motion vector candidate index used to identify the position of the selected motion vector candidate in the predicted motion vector candidate list. The inter prediction unit 121 may further output the MVD for the PU. The following describes in detail a possible implementation of the advanced motion vector prediction (AMVP) mode in FIG. 6 in this embodiment of the present invention.

In addition to performing IME and FME on the PU to generate motion information of the PU, the inter prediction unit 121 may further perform a Merge operation for each PU. When inter prediction unit 121 performs a merge operation on a PU, inter prediction unit 121 may generate a list of predicted motion vector candidates for the PU. The list of motion vector predictor candidates for the PU may include one or more original motion vector predictor candidates and one or more additional motion vector predictor candidates derived from the one or more original motion vector predictor candidates. The initial motion vector predictor candidates in the motion vector predictor candidate list may include one or more spatial motion vector predictor candidates and temporal motion vector predictor candidates. The spatial candidate of the predicted motion vector may indicate motion information of another PU of the current picture. The temporal candidate of the predicted motion vector may be based on the motion information of the corresponding PU of a picture other than the current picture. A temporal motion vector predicted candidate may also be referred to as a temporal motion vector prediction (TMVP) mode.

After listing the predicted motion vector candidates, the inter prediction unit 121 may select one predicted motion vector candidate from the list of predicted motion vector candidates. Then, the inter prediction unit 121 may generate a PU prediction image block based on the reference block indicated by the PU motion information. In the merge mode, the motion information of the PU may be the same as the motion information indicated by the selected motion vector predictor candidate. Fig. 5, described below, is a block diagram of an example of a merge mode.

After generating the prediction image block PU through the IME and FME and generating the prediction image block PU by the merge operation, the inter prediction unit 121 may select the prediction image block generated by the FME operation or the prediction image block generated by the merge operation. In some possible implementations, inter prediction module 121 may select a prediction image block for the PU by analyzing the rate-distortion cost of the prediction image block generated by the FME operation and the prediction image block generated by the merge operation.

After the inter prediction unit 121 has selected a prediction image block from a PU generated by splitting the current CU based on each split mode (in some implementations, after the coding tree block of the CTU is split into CUs, the CU is no longer split into smaller PUs). , in which case, PU is equivalent to CU), inter prediction unit 121 may select a split mode for the current CU. In some implementations, inter prediction module 121 may select a split mode for the current CU by analyzing a distortion cost of a selected PU prediction image block generated by splitting the current CU based on each split mode. The inter prediction module 121 may output a prediction image block associated with a PU that belongs to the selected partition mode to the residual generation module 102 . The inter prediction unit 121 may output to the entropy encoding unit a syntax element indicating motion information of the PU belonging to the selected partition mode.

In the diagram in Fig. 4, inter prediction module 121 includes IME modules 180A-180N (collectively referred to as "IME module 180"), FME modules 182A-182N (collectively referred to as "FME module 182"), fusion modules 184A-184N (collectively referred to as "module 184 merge"), PU mode decision modules 186A-186N (collectively referred to as "PU mode decision module 186"), and CU mode decision module 188 (and may further execute CTU to CU mode decision process).

The IME module 180, the FME module 182, and the merge module 184 can respectively perform an IME operation, an FME operation, and a merge operation on the PU of the current CU. In the diagram in Fig. 4, inter prediction module 121 is described as including a separate IME module 180, a separate FME module 182, and a separate merge module 184 for each PU in each split mode for the CU. In another possible implementation, inter prediction module 121 does not include a separate IME module 180, a separate FME module 182, or a separate merge module 184 for each PU in each split mode for the CU.

As shown in the diagram in Fig. 4, the IME module 180A, the FME module 182A, and the merge module 184A can respectively perform an IME operation, an FME operation, and a merge operation on a PU generated by splitting a CU based on a 2N × 2N split mode. The PU mode decision module 186A may select one of the prediction image blocks generated by the IME module 180A, the FME module 182A, and the fusion module 184A.

The IME module 180B, the FME module 182B, and the merge module 184B can respectively perform an IME operation, an FME operation, and a merge operation on the left PU generated by splitting the CU based on the N × 2N split mode. The PU mode decision module 186B may select one of the prediction image blocks generated by the IME module 180B, the FME module 182B, and the merge module 184B.

The IME module 180C, the FME module 182C, and the merge module 184C can respectively perform an IME operation, an FME operation, and a merge operation on the right PU generated by splitting the CU based on the N × 2N split mode. The PU mode decision module 186C may select one of the prediction image blocks generated by the IME module 180C, the FME module 182C, and the fusion module 184C.

The IME module 180N, the FME module 182N, and the merge module 184N can respectively perform an IME operation, an FME operation, and a merge operation on the bottom right PU generated by splitting the CU based on the N×N split mode. The PU mode decision module 186N may select one of the prediction image blocks generated by the IME module 180N, the FME module 182N, and the fusion module 184N.

The PU decision module 186 may select a prediction image block by analyzing the rate distortion cost for a plurality of possible prediction image blocks, and select the prediction image block that provides the optimal rate distortion cost in a given decoding scenario. For example, for a bandwidth limited application, PU decision module 186 may prefer to select a prediction image block for which the compression ratio has been increased, and for another application, PU decision module 186 may prefer to select a prediction image block for which quality has been improved. restored video. After the PU mode decision module 186 selects prediction image blocks for the PUs in the current CU, the CU mode decision module 188 selects the partition mode for the current CU and outputs the prediction image block and PU motion information that belongs to the selected partition mode.

Fig. 5 is a flowchart for implementing a merge mode according to an embodiment of the present invention. A video encoder (eg, video encoder 100) may perform a merge operation 201 . The merge operation 201 may include: S202: generate a list of candidates for the current prediction block. S204: generate a video prediction block associated with a candidate in the candidate list. S206: select a candidate from the list of candidates. S208: bring out the candidate. The candidate is a motion vector candidate or a motion information candidate.

In another possible implementation, the video encoder may perform a merge operation other than merge operation 201 . For example, in another possible implementation, the video encoder may perform a merge operation. In this case, the video encoder performs more or fewer steps than the steps of the merge operation 201, or steps other than the steps of the merge operation 201. In another possible implementation, the video encoder may perform the steps of merge operation 201 in different orders or in parallel. The encoder may further perform a merge operation 201 with a PU encoded based on a skip mode.

After the video encoder starts merge operation 201, the video encoder may generate a predicted motion vector candidate list for the current PU (S202). The video encoder may generate a predicted motion vector candidate list for the current PU in various ways. For example, the video encoder may generate a list of predicted motion vector candidates for the current PU using one of the exemplary techniques described below with reference to FIG. 8- fig. 12.

As described above, the list of predicted motion vector candidates for the current PU may include temporal motion vector predicted candidates. The temporal motion vector predictor candidate may indicate motion information corresponding to the temporal (co-located) PU. The aligned PU may be spatially located at the same position as the current image frame PU, but as a reference image instead of the current image. In the present invention, a reference picture including a corresponding temporal PU may be referred to as a corresponding reference frame. In the present invention, a reference picture index of a corresponding reference frame may be referred to as a corresponding reference picture index. As described above, the current picture may be associated with one or more reference picture lists (eg, list 0 and list 1). The reference picture index may indicate the reference picture by indicating the position of the reference picture in the reference picture list. In some possible implementations, the current picture may be associated with a combined list of reference pictures.

In some video encoders, the associated reference picture index is a PU reference picture index that spans the reference index source position associated with the current PU. In these video encoders, the reference index source position associated with the current PU is adjacent to the current PU to the left of the current PU or above the current PU. In the present invention, if an image block associated with a PU includes a specific position, the PU may "cover" the specific position. In these video encoders, the video encoder may use reference picture index 0 if the reference index source position is not available.

However, in some examples, the reference index source position associated with the current PU is within the current CU. In these examples, a PU that covers the source position of the reference index associated with the current PU may be considered available if the PU is above or to the left of the current CU. In this case, the video encoder may need access motion information of another PU of the current CU to determine a reference picture including the aligned PU. Thus, these video encoders can use motion information (namely, a reference picture index) of a PU belonging to the current CU to generate a temporal predicted motion vector candidate for the current PU. In other words, these video encoders can use the motion information of the PU belonging to the current CU to generate a temporal candidate of the predicted motion vector. Thus, the video encoder may not be able to generate in parallel a predicted motion vector candidate list for the current PU and PU that spans the reference index source position associated with the current PU.

According to the technology in the present invention, the video encoder can explicitly set the corresponding reference picture index without referring to the reference picture index of any other PU. Thus, the video encoder can generate predicted motion vector candidate lists for the current PU and another PU of the current CU in parallel. The video encoder explicitly sets the corresponding key frame index associated with the reference picture index not based on the motion information of any other PU of the current CU. In some possible embodiments in which the video encoder explicitly sets its associated reference picture index, the video encoder can always set a corresponding fixed, given reference picture index and a given reference picture index (eg, 0). Thus, the video encoder can generate a temporal motion vector predicted candidate based on the motion information of the aligned PU in the reference frame indicated by the given reference picture index, where the temporal motion vector predicted candidate can be included in the list of predicted motion vector candidates for the current CU.

In an exemplary implementation in which the video encoder explicitly sets the associated reference picture index, the video encoder may explicitly signal the associated reference picture index in a syntax structure (eg, picture, segment header, APS, or other syntax structure). In this workable implementation, the video encoder may signal the associated reference picture index of each LCU (namely, CTU), CU, PU, TU, or other sub-unit type to the decoder side. For example, the video encoder may signal that the associated reference picture index of each CU PU is "1".

In some possible embodiments, the associated reference picture index may be set indirectly rather than explicitly. In these exemplary implementations, the video encoder may generate each temporal motion vector candidate in the current CU's PU predicted motion vector candidate list using the PU's motion information as a reference picture indicated by the PU's reference picture index that spans positions outside the current CU even if these positions are not strictly adjacent to the current PU.

After generating the predicted motion vector candidate list for the current PU, the video encoder may generate a prediction picture block associated with the predicted motion vector candidate in the predicted motion vector candidate list (S204). The video encoder may determine the motion information of the current PU based on the motion information of the specified motion vector prediction candidate, and then generate a prediction image block based on one or more reference blocks indicated by the current PU's motion information to generate a prediction image block associated with the predicted motion vector candidate. The video encoder may then select one predicted motion vector candidate from the list of predicted motion vector candidates (S206). The video encoder may select the predicted motion vector candidate in various ways. For example, the video encoder may select one predicted motion vector candidate by analyzing the distortion cost for each of the prediction image blocks associated with the predicted motion vector candidates.

After selecting a predicted motion vector candidate, the video encoder may output an index of the predicted motion vector candidate (S208). The motion vector predictor candidate index may indicate the position of the selected motion vector candidate in the predicted motion vector candidate list. In some possible implementations, the predicted motion vector candidate index may be represented as "merge_idx".

Fig. 6 is a flowchart for implementing an advanced motion vector prediction (AMVP) mode according to an embodiment of the present invention. A video encoder (eg, video encoder 100) may perform AMVP operation 210. AMVP operation 210 may include: S211: generate one or more motion vectors for the current prediction block. S212: generate a video prediction block for the current prediction block. S213: generate a list of candidates for the current prediction block. S214: generate motion vector difference. S215: select a candidate from the list of candidates. S216: Output a reference picture index, a candidate index, and a motion vector difference that is used for the selected candidate. The candidate is a motion vector candidate or a motion information candidate.

After the start of AMVP operation 210 by the video encoder, the video encoder may generate one or more motion vectors for the current PU (S211). The video encoder may perform integer motion estimation and fractional motion estimation to generate a motion vector for the current PU. As described above, the current picture may be associated with two reference picture lists (List 0 and List 1). If the current PU is predicted unidirectionally, the video encoder may generate a list-0 motion vector or a list-1 motion vector for the current PU. A list 0 motion vector may indicate spatial movement between an image block corresponding to the current PU and a reference block as a reference image in list 0. A list 1 motion vector may indicate spatial movement between an image block corresponding to the current PU and a reference block in reference picture in list 1. If the current PU is bidirectionally predicted, the video encoder can generate a list 0 motion vector and a list 1 motion vector for the current PU.

After generating one or more motion vectors for the current PU, the video encoder may generate a prediction picture block for the current PU (S212). The video encoder may generate a prediction picture block for the current PU based on one or more reference blocks indicated by one or more motion vectors of the current PU.

Additionally, the video encoder may generate a predicted motion vector candidate list for the current PU (S213). The video encoder may generate a predicted motion vector candidate list for the current PU in various ways. For example, the video encoder may generate a list of predicted motion vector candidates for the current PU using one or more of the possible implementations described below with reference to FIG. 8-fig. 12. In some possible implementations, when the video encoder generates the predicted motion vector candidate list in AMVP operation 210, the predicted motion vector candidate list may be limited to two predicted motion vector candidates. In contrast, when the video encoder generates the predicted motion vector candidate list in a merge operation, the predicted motion vector candidate list may include more predicted motion vector candidates (eg, five predicted motion vector candidates).

After generating the predicted motion vector candidate list for the current PU, the video encoder may generate one or more motion vector differences (MVDs) for each predicted motion vector candidate in the predicted motion vector candidate list (S214). The video encoder may determine the difference between the motion vector indicated by the predicted motion vector candidate and the corresponding motion vector of the current PU to generate the motion vector difference for the predicted motion vector candidate.

If the current PU is predicted unidirectionally, the video encoder may generate a single MVD for each predicted motion vector candidate. If the current PU is bidirectionally predicted, the video encoder may generate two MVDs for each predicted motion vector candidate. The first MVD may indicate a difference between the motion vector indicated by the predicted motion vector candidate and the list 0 motion vector of the current PU. The second MVD may indicate a difference between the motion vector indicated by the predicted motion vector candidate and the list 1 motion vector of the current PU.

The video encoder may select one or more predicted motion vector candidates from a list of predicted motion vector candidates (S215). The video encoder may select one or more predicted motion vector candidates in various manners. For example, the video encoder may select a predicted motion vector candidate that matches, with minimal error, the motion vector associated with the motion vector to be encoded. This can reduce the number of bits needed to represent the motion vector difference for a predicted motion vector candidate.

After selecting one or more motion vector predictor candidates, the video encoder may output one or more reference picture indexes for the current PU, one or more motion vector predictor candidate indices for the current PU, and one or more motion vector differences for one or more selected motion vector predictor candidates ( S216).

As an example, in which the current picture is associated with two reference picture lists (list 0 and list 1) and the current PU is unidirectionally predicted, the video encoder may output a reference picture index ("ref_idx_10") for list 0 or a reference picture index ("ref_idx_11") for list 1. The video encoder may further output a predicted motion vector candidate index ("mvp_10_flag") indicating the position of the selected motion vector candidate for list 0 of the current PU's motion vector in the predicted motion vector candidate list. Alternatively, the video encoder may output a predicted motion vector candidate index ("mvp_11_flag") that indicates the position of the selected motion vector of the list 1 predicted motion vector candidate of the current PU in the predicted motion vector candidate list. The video encoder may further derive the MVD for the list 0 motion vector or the list 1 motion vector of the current PU.

As an example, in which the current picture is associated with two reference picture lists (list 0 and list 1) and the current PU is bidirectionally predicted, the video encoder may output a reference picture index ("ref_idx_10") for list 0 and a reference picture index ("ref_idx_11") for list 1. The video encoder may further output a predicted motion vector candidate index ("mvp_10_flag") indicating the position of the selected motion vector candidate for list 0 of the current PU's motion vector in the predicted motion vector candidate list. In addition, the video encoder may output a predicted motion vector candidate index ("mvp_11_flag") that indicates the position of the selected motion vector candidate for the current PU list 1 motion vector in the predicted motion vector candidate list. The video encoder may further output an MVD for the list 0 motion vector of the current PU and an MVD for the list 1 motion vector of the current PU.

Fig. 7 is a flowchart of a motion compensation implementation performed by a video decoder (eg, video decoder 30) according to an embodiment of the present invention.

When the video decoder performs motion compensation operation 220, the video decoder may receive an indication used for the selected motion vector prediction candidate of the current PU (S222). For example, the video decoder may receive a predicted motion vector candidate index indicating the position of the selected motion vector candidate in the list of predicted motion vector candidates for the current PU.

If the motion information of the current PU is encoded based on the AMVP mode and the current PU is bidirectionally predicted, the video decoder may receive a first predicted motion vector candidate index and a second predicted motion vector candidate index. The first predicted motion vector candidate index indicates the position of the selected motion vector prediction candidate for the list 0 motion vector of the current PU in the predicted motion vector candidate list. The second predicted motion vector candidate index indicates the position of the selected motion vector prediction candidate for the motion vector of list 1 of the current PU in the predicted motion vector candidate list. In some possible implementations, a single syntax element may be used to identify two motion vector predictor candidate indices.

Additionally, the video decoder may generate a predicted motion vector candidate list for the current PU (S224). The video decoder may generate a predicted motion vector candidate list for the current PU in various ways. For example, the video decoder may generate a predicted motion vector candidate list for the current PU using the techniques described below with reference to FIG. 8-fig. 12. When the video decoder generates a temporal motion vector predicted candidate for the motion vector predicted candidate list, the video decoder may explicitly or implicitly set a reference picture index that identifies the reference picture including the aligned PU, as described above with reference to FIG. five.

After generating a list of predicted motion vector candidates for the current PU, the video decoder may determine the motion information of the current PU based on the motion information indicated by one or more selected candidate motion vectors in the list of predicted motion vector candidates for the current PU (S225). For example, if the motion information of the current PU is encoded based on the merge mode, the motion information of the current PU may be the same as the motion information indicated by the selected motion vector prediction candidate. If the motion information of the current PU is encoded based on the AMVP mode, the video decoder may recover one or more motion vectors of the current PU using one or more motion vectors indicated by one or more selected motion vector predictor candidates and one or more MVDs indicated in the bitstream. The reference frame index and prediction direction indicator of the current PU may be the same as one or more reference picture indices and prediction direction indicators of one or more selected motion vector predictor candidates. After determining the motion information of the current PU, the video decoder may generate a prediction picture block for the current PU based on one or more reference blocks indicated by the motion information of the current PU (S226).

Fig. 8 is a diagram of an example of a coding unit (CU) and an adjacent image block associated with a coding unit (CU) according to an embodiment of the present invention. Fig. 8 is a diagram for describing CU 250 and exemplary motion vector predictor candidates at positions 252A through 252E associated with CU 250. In the present invention, the predicted motion vector candidates at positions 252A-252E may be collectively referred to as predicted motion vector candidates at position 252. The motion vector candidate at position 252 represents a spatial motion vector predictor candidate that is in the same picture as CU 250. The motion vector predictor candidate at position 252A is located to the left of CU 250. The motion vector predictor candidate at position 252B is located above CU 250. The predicted motion vector candidate at 252C is located in the upper right corner of CU 250. The predicted motion vector candidate at 252D is located in the lower left corner of CU 250. The predicted motion vector candidate at 252E is located in the upper left corner of CU 250. FIG. 8 shows a schematic implementation of the manner in which the inter prediction module 121 and the motion compensation module can generate predicted motion vector candidate lists. Next, the implementation is explained based on the inter prediction unit 121 . However, it should be understood that the motion compensation module may implement the same technology and therefore generate the same list of predicted motion vector candidates.

Fig. 9 is a flowchart of an implementation of predicted motion vector candidate listing according to an embodiment of the present invention. The technology in Fig. 9 is described based on a list of five predicted motion vector candidates, but the techniques described in this specification may alternatively be used with a list having a different size. Each of the five predicted motion vector candidates may have an index (eg, 0 to 4). The technology in Fig. 9 is described based on a common video codec. The general video codec may be, for example, a video encoder (eg, video encoder 100) or a video decoder (eg, video decoder 30).

To reconstruct the predicted motion vector candidate list according to the implementation of FIG. 9, the video decoder first considers four spatial motion vector predictor candidates (902). The four spatial motion vector predictor candidates may include the predicted motion vector candidates at positions 252A, 252B, 252C, and 252D. The four spatial motion vector candidates may correspond to motion information of four PUs that are located in the same picture as the current CU (eg, CU 250). The video decoder may consider the four spatial predicted motion vector candidates in the list in that order. For example, the predicted motion vector candidate at position 252A may be considered first. If the predicted motion vector candidate at 252A is available, index 0 may be assigned to the predicted motion vector candidate at 252A. If the predicted motion vector candidate at position 252A is not available, the video decoder may skip adding the predicted motion vector candidate at position 252A to the predicted motion vector candidate list. The position predicted motion vector candidate may not be available for various reasons. For example, if a position predicted motion vector candidate is not present in the current picture, a position predicted motion vector candidate may not be available. In another possible implementation, if the position predicted motion vector candidate is intra-predicted, the position predicted motion vector candidate may not be available. In another possible implementation, if the position predicted motion vector candidate is in a segment other than the segment corresponding to the current CU, the position predicted motion vector candidate may not be available.

After considering the predicted motion vector candidate at 252A, the video decoder may consider the predicted motion vector candidate at 252B. If the predicted motion vector candidate at position 252B is available and is different from the predicted motion vector candidate at position 252A, the video decoder may add the predicted motion vector candidate at position 252B to the predicted motion vector candidate list. In this particular context, the term "same" or "different" means that the motion information associated with the predicted motion vector candidates at the positions is the same or different. Therefore, if two position motion vector predictor candidates have the same motion information, the two position motion vector predictor candidates are considered to be the same; or, if two position motion vector prediction candidates have different motion information, then the two position motion vector prediction candidates are considered different. If the predicted motion vector candidate at position 252A is not available, the video decoder may assign an index of 0 to the predicted motion vector candidate at position 252B. If the predicted motion vector candidate at position 252A is available, the video decoder may assign an index of 1 to the predicted motion vector candidate at position 252B. If the predicted motion vector candidate at position 252B is not available or matches the predicted motion vector candidate at position 252A, the video decoder skips adding the predicted motion vector candidate at position 252B to the predicted motion vector candidate list.

Similarly, the video decoder considers the predicted motion vector candidate at position 252C to determine whether to add the predicted motion vector candidate at position 252C to the list. If the predicted motion vector candidate at position 252C is available and is different from the predicted motion vector candidates at positions 252B and 252A, the video decoder may assign the next available index to the predicted motion vector candidate at position 252C. If the predicted motion vector candidate at position 252C is not available or matches at least one of the predicted motion vector candidates at positions 252A and 252B, the video decoder skips adding the predicted motion vector candidate at position 252C to the predicted motion vector candidate list. The video decoder then considers a predicted motion vector candidate at position 252D. If the predicted motion vector candidate at position 252D is available and is different from the predicted motion vector candidates at positions 252A, 252B, and 252C, the video decoder may assign the next available index to the predicted motion vector candidate at position 252D. If the predicted motion vector candidate at position 252D is not available or matches at least one of the predicted motion vector candidates at positions 252A, 252B, and 252C, the video decoder skips adding the predicted motion vector candidate at position 252D to the predicted motion vector candidate list. The above implementation generally describes an example in which the predicted motion vector candidates 252A-252D are examined to determine whether to add the predicted motion vector candidates 252A-252D to the predicted motion vector candidate list. However, in some implementations, all of the predicted motion vector candidates 252A-252D may be first added to the predicted motion vector candidate list, and then the duplicate predicted motion vector candidate is removed from the predicted motion vector candidate list.

After the video decoder considers the first four spatial motion vector predicted candidates, the list of predicted motion vector candidates may include four spatial predicted motion vector candidates, or the list may include less than four spatial predicted motion vector candidates. If the list includes four spatial motion vector predicted candidates (904, yes), the video decoder considers a temporal motion vector predicted candidate (906). The predicted motion vector temporal candidate may correspond to motion information of a co-located PU image other than the current image. If a temporal predicted motion vector candidate is available and different from the first four spatial predicted motion vector candidates, the video decoder assigns index 4 to the temporal predicted motion vector candidate. If the temporal predicted motion vector candidate is not available or matches one of the first four spatial predicted motion vector candidates, the video decoder skips adding the temporal predicted motion vector candidate to the list of predicted motion vector candidates. Therefore, after the video decoder considers the temporal motion vector predicted candidate (906), the list of predicted motion vector candidates may include five predicted motion vector candidates (the first four spatial motion vector predicted candidates considered at 902 and the temporal predicted vector candidate considered motion vector candidates in step 906) or may include four predicted motion vector candidates (the first four predicted motion vector candidates are considered in step 902). If the list of predicted motion vector candidates includes five predicted motion vector candidates (908, yes), the video decoder completes the list.

If the list of predicted motion vector candidates includes four predicted motion vector candidates (908, none), the video decoder may consider a fifth predicted motion vector candidate (910). The fifth spatial motion vector predictor candidate may (for example) correspond to the predicted motion vector candidate at position 252E. If the predicted motion vector candidate at position 252E is available and different from the predicted motion vector candidates at positions 252A, 252B, 252C, and 252D, the video decoder may add the fifth spatial motion vector candidate to the list of predicted motion vector candidates and assign index 4 to the fifth spatial predicted vector candidate. movement. If the predicted motion vector candidate at position 252E is unavailable or matches one of the predicted motion vector candidates at positions 252A, 252B, 252C, and 252D, the video decoder may skip adding the predicted motion vector candidate at position 252 to the predicted motion vector candidate list. Therefore, after consideration (910) of the fifth spatial motion vector predicted candidate, the list may include five motion vector predicted candidates (the first four spatial motion vector predicted candidates considered at 902 and the fifth spatial motion vector predicted candidate considered at 910), or may include four predicted motion vector candidates (the first four spatial motion vector predicted candidates are considered in step 902).

If the motion vector predicted candidate list includes five motion vector predicted candidates (912, YES), the video decoder ends generation of the motion vector predicted candidate list. If the motion vector predicted candidate list includes four motion vector predicted candidates (912, none), the video decoder adds an artificially generated motion vector predicted candidate (914) until the list includes five motion vector predicted candidates (916, Yes).

If the list includes less than four spatial motion vector predicted candidates (904, none), after the video decoder considers the first four spatial motion vector predicted candidates, the video decoder may consider a fifth spatial motion vector predicted candidate (918). The fifth spatial motion vector predictor candidate may (for example) correspond to the predicted motion vector candidate at position 252E. If the predicted motion vector candidate at position 252E is available and is different from the existing predicted motion vector candidates in the predicted motion vector candidate list, the video decoder may add the fifth spatial motion vector candidate to the predicted motion vector candidate list, and assign the next available index to the fifth spatial predicted motion vector candidate. motion vector. If the predicted motion vector candidate at position 252E is unavailable or matches one of the existing predicted motion vector candidates in the predicted motion vector candidate list, the video decoder may skip adding the predicted motion vector candidate at position 252E to the predicted motion vector candidate list. The video decoder may then consider a temporal candidate of the predicted motion vector (920). If a temporal motion vector predicted candidate is available and different from an existing predicted motion vector candidate in the predicted motion vector candidate list, the video decoder may add the temporal motion vector candidate to the predicted motion vector candidate list and assign the next available index to the temporal predicted motion vector candidate. If the temporal predicted motion vector candidate is not available or is the same as one of the existing predicted motion vector candidates in the predicted motion vector candidate list, the video decoder may skip adding the temporal predicted motion vector candidate to the predicted motion vector candidate list.

If the list of predicted motion vector candidates includes five predicted motion vector candidates (922, YES) after considering the fifth spatial motion vector candidate (at 918) and the temporal motion vector predicted candidate (at 920), the video decoder ends generating list of predicted motion vector candidates. If the motion vector predicted candidate list includes less than five motion vector predicted candidates (922, none), the video decoder adds an artificially generated motion vector predicted candidate (914) until the list includes five motion vector predicted candidates (916 , Yes).

According to the techniques in the present invention, an additional predicted motion vector merge candidate can be artificially generated after the spatial predicted motion vector candidate and the temporal predicted motion vector candidate, so that the size of the predicted motion vector merge candidate list is fixed, and the predicted motion vector merge candidate list includes a specified number (eg, five in the above exemplary implementation in FIG. 9) of predicted motion vector fusion candidates. The additional predicted motion vector merge candidate may include examples of a combined dual predicted motion vector merge candidate (motion vector predicted candidate 1), a scaled dual predicted motion vector merge candidate (motion vector predicted candidate 2), and a predicted merge/AMVP candidate. zero motion vector motion vector (predicted motion vector candidate 3).

Fig. 10 is a diagram of an example of adding a combined motion vector candidate to a merge mode predicted motion vector candidate list according to an embodiment of the present invention. The combined dual-predictive merge motion vector candidate can be generated by combining the original merge motion vector candidates. In particular, two initial motion vector predictor candidates (which have mvL0 and refIdxL0 or mvL1 and refIdxL1) may be used to generate a dual prediction merge motion vector candidate. In FIG. 10, two predicted motion vector candidates are included in the list of initial predicted motion vector fusion candidates. The prediction type of one motion vector predictor candidate is unified prediction using list 0, and the prediction type of the other predicted motion vector candidate is unified prediction using list 1. In this exemplary implementation, mvL0_A and ref0 are obtained from list 0, and mvL1_B and ref0 are obtained from list 1. Next, a dual predicted motion vector merge candidate (which has mvL0_A and ref0 in list 0 and mvL1_B and ref0 in list 1) can be generated, and it is checked whether the dual predicted motion vector merge candidate differs from the existing predicted vector candidate. movements in the candidate list of the predicted motion vector. If the dual predicted motion vector merge candidate is different from the existing predicted motion vector candidate, the video decoder may add the dual predicted motion vector merge candidate to the list of predicted motion vector candidates.

Fig. 11 is a diagram of adding a scaled motion vector candidate to a merge mode predicted motion vector candidate list according to an embodiment of the present invention. A scaled dual predicted motion vector merge candidate can be generated by scaling the original predicted motion vector merge candidate. In particular, one original motion vector predictor candidate (which has mvLX and refIdxLX) may be used to generate a dual predicted motion vector merge candidate. In a possible implementation in FIG. 11, two predicted motion vector candidates are included in the list of initial predicted motion vector fusion candidates. The prediction type of one motion vector predictor candidate is unified prediction using list 0, and the prediction type of the other predicted motion vector candidate is unified prediction using list 1. In this exemplary implementation, mvL0_A and ref0 can be obtained from list 0, ref0 can be copied to list 1 and is designated as reference index ref0'. Then mvL0'_A can be calculated by scaling mvL0_A having ref0 and ref0'. The scaling may depend on the POC (Picture Order Count) distance. Then, a dual predicted motion vector merge candidate (which has mvL0_A and ref0 in list 0 and mvL0'_A and ref0' in list 1) can be generated, and it is determined whether the dual predicted motion vector merge candidate is repeated. If the dual predicted motion vector merge candidate is not repeated, the dual predicted motion vector merge candidate may be added to the predicted motion vector merge candidate list.

Fig. 12 is a diagram of an example of adding a null motion vector to a predicted motion vector merge candidate list according to an embodiment of the present invention. A null vector predicted motion vector merge candidate can be generated by concatenating the null vector and a reference index that can be referenced. If the null vector predicted motion vector merge candidate is not repeated, the null vector predicted motion vector merge candidate may be added to the predicted motion vector merge candidate list. The motion information of each generated predicted motion vector merge candidate may be compared with the motion information of a previous predicted motion vector candidate in the list.

In an exemplary implementation, if a newly generated predicted motion vector candidate differs from an existing predicted motion vector candidate in the predicted motion vector candidate list, the generated predicted motion vector candidate is added to the predicted motion vector merge candidate list. The process of determining whether a motion vector prediction candidate differs from an existing motion vector prediction candidate in the motion vector prediction candidate list is sometimes referred to as pruning. By pruning, each newly generated predicted motion vector candidate can be compared with an existing predicted motion vector candidate in the list. In some possible implementations, the pruning operation may include: comparing one or more new predicted motion vector candidates with an existing predicted motion vector candidate in the list of predicted motion vector candidates, and skipping adding a new predicted motion vector candidate that matches the predicted motion vector candidate in the list. candidates for the predicted motion vector. In some other possible implementations, the pruning operation may include: adding one or more new predicted motion vector candidates to the list of predicted motion vector candidates, and then removing the duplicate predicted motion vector candidate from the list.

The following describes several implementations of extrinsic prediction. The first predetermined algorithm and the second predetermined algorithm in the present invention may include one or more inter prediction implementations.

Inter-prediction uses temporal correlation between images to obtain motion-compensation prediction (MCP) for a block of image samples.

For block-based MCP, the video image is divided into rectangular blocks. Assuming that uniform motion occurs within one block and that the moving objects are larger than one block, for each block, a corresponding block in the previously decoded image can be found that serves as a prediction value. When using the translational motion model, the position of a block in a previously decoded image is indicated by a motion vector (Δx, Δy) where Δx indicates a horizontal offset relative to the position of the current block and Δy indicates a vertical offset relative to the position of the current block. The motion vector (Δx, Δy) may have a fractional sampling precision to more accurately capture the movement of the reference object. When the corresponding motion vector has fractional sampling precision, interpolation is applied on the reference picture to output a prediction signal. The previously decoded picture is called as a reference picture, and is denoted by a reference index Δt corresponding to the reference picture list. These translational motion model parameters, namely motion vector and reference index, are hereinafter referred to as motion data. In current video coding standards, two types of inter prediction are allowed, namely unified prediction and bidirectional prediction.

In the case of bidirectional prediction, two sets of motion data (Δx0, Δy0, Δt0 and Δx1, Δy1, Δt1) are used to generate two MCPs (possibly from different images), which are then combined to obtain the final MCP. By default, this is done by averaging, but in the case of weighted prediction, different weights can be applied to each MCP, for example, to compensate for the gradual disappearance of the image. Reference pictures that can be used in bidirectional prediction are stored in two separate lists, namely list 0 and list 1. To limit the memory bandwidth in a segment that provides bidirectional prediction, the HEVC standard limits the PU to either 4 x 8 luma prediction blocks or 8×4 to use only unified prediction. The motion data is output to the encoder using a motion estimation process. Motion estimation is not specified in the video standards, so that different encoders may use different complexity and quality trade-offs during encoder implementation.

The movement data of a block is correlated with a neighboring block. To use this correlation, the motion data is not encoded directly in the bitstream, but is predictively encoded based on the adjacent motion data. HEVC uses two concepts for this. Motion vector prediction coding has been improved in HEVC by introducing a new tool called advanced motion vector prediction (AMVP), in which the best prediction value for each motion block is passed to the decoder. Additionally, a new technique called inter prediction block fusion is used to extract all block motion data from neighboring blocks. Thus, direct mode and skip mode are replaced in H.264/AVC.

Advanced Motion Vector Prediction

As described in previous video coding standards, the HEVC motion vector is encoded as the difference of the motion vector prediction value (MVP) based on the horizontal (x) component and the vertical (y) component. The two motion vector difference (MVD) components are calculated according to equations (1.1) and (1.2).

MVD _X = ∆x - MVP _X (1.1)

MVD _Y = Δy - MVP _Y (1.2)

The motion vector of the current block is typically correlated with the motion vector of a neighboring block in the current picture or a previously encoded picture. This is associated with the fact that the adjacent block is likely to correspond to the same moving object with similar motion, and the motion of the object is unlikely to change dramatically over time. Therefore, using the motion vector of an adjacent block as a prediction value reduces the signaled motion vector difference. The MVP is typically obtained from an already decoded motion vector of a spatially adjacent block or a temporally adjacent block in a co-located picture. In some cases, motion vector zero may alternatively be used as an MVP. In H.264/AVC, this is implemented by performing a componentwise median of three spatial adjacent motion vectors. Using this approach, signaling the prediction value is not required. in H.264/AVC, only the temporal MVP from the merged image is considered in the so-called temporal direct mode. H.264/AVC direct mode is also used to receive motion data other than motion vector.

In HEVC, the approach to implicit MVP acquisition has been replaced by a technology known as motion vector contention. The technology explicitly signals which MVP from the MVP list is used to get the motion vector. The variable coding quadtree block structure in HEVC can result in a single block having multiple neighbor blocks with motion vectors to serve as potential MVP candidates. The original Advanced Motion Vector Prediction (AMVP) structure included five MVPs from three different classes of prediction values: three motion vectors from spatial neighbors, a median of three spatial prediction values, and a scaled motion vector from an aligned temporal neighbor block. In addition, the list of prediction values has been modified by reordering to put the most likely motion prediction value in the first position and by removing a redundant candidate to ensure minimal signaling overhead. Significant simplifications of the AMVP structure are then developed, such as removing the mean of the prediction, reducing the number of candidates in the list from five to two, fixing the order of the candidates in the list, and reducing the number of redundancy checks. The final build of the AMVP candidate roster includes the following two MVP candidates: a. up to two spatial candidate MVPs derived from five spatial neighboring blocks; b. one temporal candidate MVP derived from two co-located temporal blocks when both spatial candidate MVPs are unavailable or identical; and c. zero motion vector when the spatial candidates, the temporal candidate, or both the spatial candidates and the temporal candidate are unavailable.

As already mentioned, two spatial MVP candidates A and B are derived from five spatial neighbor blocks. The positions of spatial block candidates are the same for both AMVP and inter prediction block merging. For candidate A, the motion data of the two blocks A0 and A1 in the lower left corner are taken into account in a two-pass approach. The first pass checks to see if any of the block candidates includes a reference index that is equal to the reference index of the current block. The first motion vector that is found is used as candidate A. When all the reference indices from A0 and A1 point to a different reference picture than the reference index of the current block, its associated motion vector cannot be used as is. Therefore, in the second pass, the motion vector needs to be scaled based on the temporal distance between the reference picture candidate and the current reference picture. Equation (1.3) shows how the motion vector candidate mvcand is scaled based on the scaling factor. The scaling factor is calculated based on the temporal distance between the current picture and the block candidate reference picture td and the temporal distance between the current picture and the current block reference picture tb. The temporal distance is expressed in terms of the difference between the picture order count (POC) values, which determines the order in which the pictures are displayed. The scaling operation is basically the same as the scheme used for the temporary direct mode in H.264/AVC. This factorization allows the scaling factor to be precomputed at the segment level, since this factorization depends only on the structure of the reference picture list reported in the segment header. It should be noted that MV scaling is only performed when both the current reference picture and the reference picture candidate are short-term reference pictures. The parameter td is defined as the POC difference between the co-located picture and the reference picture for the co-located block candidate.

mv = sign (mvcand • ScaleFactor) ((| mvcand • ScaleFactor | + 27) >> 8) (1.3)

ScaleFactor = clip (–212, 212 - 1, (tb • tx + 25) >> 6) (1.4)

(1.5)

(1.5)

For candidate B, candidates from B0 to B2 are sequentially checked in the same way as when checking A0 and A1 on the first pass. However, the second pass is only performed when blocks A0 and A1 do not include any motion information, in other words, when blocks A0 and A1 are not available or are encoded by intra picture prediction. Then, if candidate A is found, candidate A is set equal to the unscaled candidate B and candidate B is set equal to the second unscaled or scaled variant of candidate B. On the second pass, a search is performed to obtain the unscaled MV and the scaled MV from candidates B0 to B2. In general, this structure allows A0 and A1 to be processed independently of B0, B1 and B2. When inferring B, only the availability of both A0 and A1 should be considered in order to search for a scaled MV or an additional unscaled MV from B0 to B2. This relationship is acceptable provided that it significantly reduces the complex motion vector scaling operation for candidate B. Reducing the amount of motion vector scaling represents a significant reduction in complexity in the process of deriving a motion vector prediction value.

In HEVC, the blocks in the lower right corner and in the center of the current block were determined to be the most suitable blocks to provide a good temporal motion vector prediction value (TMVP). Among these candidates, C0 represents the bottom right neighbor block and C1 represents the center block. Here again, motion data C0 is considered first. If C0 motion data is not available, the motion data of the co-located block candidate at the center is used to obtain the temporary MVP of candidate C. The C0 motion data is also considered unavailable when the associated PU belongs to a CTU that is outside the current CTU row. This minimizes the memory bandwidth requirements for storing co-located motion data. Unlike spatial MVP candidates, motion vectors can refer to the same reference picture, motion vector scaling is mandatory for TMVP. Therefore, the same scaling operation is used as for the spatial MVP.

Although H.264/AVC direct temporal mode always refers to the first reference picture in the second reference picture list, list 1, and is only allowed in bidirectional segment prediction, HEVC provides the ability to specify for each picture which reference picture treated as a combined image. This is accomplished by signaling a list of aligned reference pictures and a reference picture index in the segment header, and by requesting syntax elements in all segments in the picture to specify the same reference picture.

Because the interim MVP candidate introduces an additional dependency, the use of the interim MVP candidate may need to be disabled due to fault tolerance. In H.264/AVC, it is possible to disable temporal direct mode for a bidirectional segment in the segment header (direct_spatial_mv_pred_flag). The HEVC syntax extends this signaling to allow TMVP to be disabled at the sequence or image level (sps/slice_temporal_mvp_enabled_flag). Although the segment header reports a flag, bitstream matching requires the flag value to be the same for all segments in the same picture. Since the image level flag signaling is dependent on the SPS flag, the image level flag signaling in the PPS may introduce a parsing dependency between the SPS and the PPS. Another advantage of this segment header signaling is that if only the value of this flag in the PPS needs to be changed without changing the other parameter, there is no need to send a second PPS.

In general, motion data signaling in HEVC is similar to motion data signaling in H.264/AVC. The inter prediction syntax element, inter_pred_idc, signals whether reflist 0, reflist 1, or both reflist 0 and reflist 1 are used. image, ref_idx_l0/1 and MV (Δx, Δy) is represented by an index in MVP, mvp_l0/1_flag and MVD MVP. The newly introduced flag in the segment header, mvd_l1_zero_flag, indicates whether the MVD for the second reference picture list is zero and therefore not signaled in the bitstream. When the motion vector is completely restored, the final clip operation ensures that the value of each component of the final motion vector is always in the range -2 ¹⁵ to 2 ¹⁵ -1 inclusive.

Merging a block (Block) of external prediction

The AMVP list only includes the motion vector for one reference list, while the merge candidate includes all motion data including whether one or two reference picture lists are used, and the reference index and motion vector for each list. In general, the list of merger candidates consists of the following candidates: a. up to four spatial merging candidates derived from five adjacent spatial blocks; b. one temporary merge candidate derived from two co-located blocks; and c. additional merge candidates including a combined dual-prediction candidate and a null motion vector candidate.

The first candidate in the merge candidate list is the spatial neighbor. A1, B1, B0, A0, and B2 are checked sequentially, and up to four candidates can be inserted into the merge list in that order.

Instead of simply checking if a neighboring block is available and includes motion information, some additional redundancy checks are performed before all of the neighboring block's motion data is used as a merge candidate. These redundancy checks can be divided into two categories for two different purposes: a. Avoid having a candidate with redundant traffic data on the list; and b. Avoid merging two sections that might be expressed by another means that might create redundant syntax.

сравнений данных движения. В случае пяти потенциальных кандидатов на пространственное слияние необходимо выполнить 10 сравнений данных движения, чтобы гарантировать, что все кандидаты в списке слияния имеют разные данные движения. Во время разработки HEVC проверки избыточных данных движения были сокращены до подмножества, так что эффективность кодирования сохраняется, и логика сравнения значительно снижается. В окончательном варианте для каждого кандидата выполняется не более двух сравнений, всего пять сравнений. Учитывая порядок {A1, B1, B0, A0, B2}, B0 проверяет только B1, A0 только A1 и B2 только A1 и B1. В варианте осуществления, в котором выполняется проверка избыточности разделения, нижний PU раздела 2N × N объединяется с верхним PU путем выбора кандидата B1. В этом случае у одного CU есть два PUs с одинаковыми данными движения. CU может также сигнализироваться как 2N × 2N CU. В целом эта проверка применяется ко всем вторым PUs прямоугольных и асимметричных разделов: 2N × N, 2N × nU, 2N × nD, N × 2N, nR × 2N и nL × 2N. Следует отметить, что для кандидатов на пространственное слияние выполняется только проверка избыточности, и данные движения копируются из кандидатов блоков как есть. Следовательно, здесь не требуется масштабирование вектора движения.When N represents the number of spatial merge candidates, the full redundancy check consists of

motion data comparisons. In the case of five potential spatial merge candidates, 10 motion data comparisons must be performed to ensure that all candidates in the merge list have different motion data. During the development of HEVC, checks for redundant motion data were reduced to a subset so that coding efficiency is maintained and comparison logic is greatly reduced. In the final version, no more than two comparisons are made for each candidate, for a total of five comparisons. Given the order {A1, B1, B0, A0, B2}, B0 checks only B1, A0 only A1, and B2 only checks A1 and B1. In an embodiment in which a partition redundancy check is performed, the lower PU of the 2N × N partition is combined with the upper PU by selecting candidate B1. In this case, one CU has two PUs with the same motion data. The CU may also be signaled as 2N × 2N CU. In general, this check applies to all second PUs of rectangular and asymmetric partitions: 2N × N, 2N × nU, 2N × nD, N × 2N, nR × 2N, and nL × 2N. It should be noted that only a redundancy check is performed for spatial merge candidates, and motion data is copied from block candidates as is. Therefore, motion vector scaling is not required here.

The motion vector candidate for temporary merge is derived in the same way as when TMVP is received. Since the merge candidate includes all motion data and TMVP is only one motion vector, obtaining all motion data depends only on the segment type. For a dual prediction segment, TMVP is output for each reference picture list. Depending on the availability of TMVP for each list, the prediction type is set to bidirectional prediction or to the list for which TMVP is available. All associated reference picture indexes are set to zero. Thus, for a unified prediction segment obtained only by TMVP for list 0, together with the reference picture index, is zero.

When at least one TMVP is available and a temporary merge candidate is added to the list, no redundancy check is performed. This makes the compilation of the merge list independent of the co-located image and therefore improves error resilience. When considering the case where the temporary merge candidate is redundant and therefore not included in the list of merge candidates, and the merged image is lost, the decoder cannot obtain the temporary candidate and therefore does not check whether the temporary candidate is redundant. This affects the indexing of all subsequent candidates.

For parse resistance, the merge candidate list has a fixed length. After adding spatial and temporal merge candidates, the list may still not reach a fixed length. To compensate for the loss in coding efficiency that occurs with adaptive list index signaling without length, an additional candidate is generated. Depending on the segment type, up to two types of candidates can be used to complete the list: a. combined candidate with double prediction; and b. candidate with zero motion vector.

In the dual prediction segment, an additional candidate can be generated based on an existing candidate by combining reference picture list 0 motion data of one candidate and reference picture list 1 motion data of another candidate. This is done by copying Δx0, Δy0, and Δt0 from one candidate, such as the first candidate, and copying Δx1, Δy1, and Δt1 from another candidate, such as the second candidate. Various combinations are predefined and are shown in Table 1.1.

Table 1.1

Combination order 0 one 2 3 4 five 6 7 eight nine 10 eleven Dx ₀ , Dy ₀ and Dt ₀ of the candidate 0 one 0 2 one 2 0 3 one 3 2 3 Dx ₁ , Dy ₁ and Dt ₁ of the candidate one 0 2 0 2 one 3 0 3 one 3 2

When the list is still empty after the combined dual prediction candidate is added or is empty for the unified prediction segment, a zero motion vector candidate is computed to complete the list. All zero motion vector candidates have one zero offset motion vector for the unified prediction segment and two zero offset motion vectors for the dual prediction segment. The pivot index is set to zero and incremented by one for each additional candidate until the maximum number of pivot indexes is reached. In this case, if there are still no other candidates, a reference index of zero is used to generate those candidates. For all additional candidates, redundancy checks are not performed, since it turned out that omission of these checks does not lead to a loss in coding efficiency.

For each PU encoded based on the inter prediction mode, merge_flag indicates that block merging is used to obtain motion data, and merge_idx is further used to determine a candidate that is in the merge list and that provides all the required motion data for the MCP. In addition to this signaling at the PU level, the number of candidates in the merge list is signaled in the segment header. Since the default value is 5, it is represented as a difference of five (five_minus_max_num_merge_cand). Thus, 5 is conveyed with a short codeword for 0. If only one candidate is used, 5 is conveyed with a longer codeword for 4. Regarding the impact on the merge candidate list building process, the overall process remains the same, although the process terminates after the list includes the maximum number of merger candidates. In the original version, the maximum value for encoding the merge index is given by the number of available spatial and temporal candidates in the list. For example, when only two candidates are available, the index can be effectively encoded as a flag. However, to analyze the merger index, it is necessary to compile the entire list of merger candidates in order to know the actual number of candidates. Assuming a neighboring block is unavailable due to a transmission error, the merge index can no longer be parsed.

An important application of the concept of block merging in HEVC is its combination with the skip mode. In previous video coding standards, the skip mode is used to indicate to the block that motion data is assumed instead of explicit signals and that the prediction residual is zero. In other words, no conversion factor is sent. In HEVC, a skip_flag is signaled at the beginning of each CU in an inter prediction segment, which implies the following: a. The CU includes only one PU (partition type 2N × 2N). b. The merge mode is used to get motion data (merge_flag is 1). c. There is no residual data in the bitstream.

The parallel merge evaluation level, which indicates that a scope is being introduced into HEVC. The merge candidate list can be obtained independently by checking whether the candidate block is located in that merge evaluation area (MER). A block of candidates in the same MER is not included in the merge candidate list. Therefore, block candidate motion data need not be available at the time of list building. When this level is 32, for example, all prediction blocks in a 32×32 region can be used to list merge candidates in parallel because all merge candidates in the same 32×32 MER are not inserted into the list. All potential merge candidates for the first PU 0 are available because all potential merge candidates are outside the first 32 × 32 MER. merge inside MER must be independent. Therefore, for example, when PU 5 is being looked up, the merge candidate is not available and therefore is not inserted in the list of merge candidates. In this case, the merge list for PU5 includes only the provisional candidate (if available) and the null MV candidate. To enable the encoder to trade off parallelism and coding efficiency, the parallel merge evaluation level is adaptive and is signaled as log2_parallel_merge_level_minus2 in the picture parameter set.

Motion vector prediction based on sub-CU

During the development of a new QTBT video coding technology, each CU can have at most one set of motion parameters for each prediction direction. The encoder considers two methods of motion vector prediction at the sub-CU level by dividing the large CU into sub-CUs and obtaining motion information for all sub-CUs of the large CU. The alternative temporal motion vector prediction (ATMVP) method allows each CU to extract a plurality of motion information sets from a plurality of smaller blocks than the current CU in the aligned reference picture. In the spatial-temporal motion vector prediction (STMVP) method, a sub-CU motion vector is obtained recursively using a temporal motion vector prediction value and an adjacent spatial motion vector.

To preserve a more accurate motion field for sub-CU motion prediction, motion compression for the reference frame is currently disabled.

Alternative temporal motion vector prediction

In an alternative temporal motion vector prediction (ATMVP) method, a temporal motion vector prediction (TMVP) for a motion vector is modified by fetching a plurality of motion information sets (including motion vectors and reference indices) from blocks , smaller than the current CU. A sub-CU is an N × N square block (where N defaults to 4).

ATMVP predicts the motion vector of the sub-CU in the CU in two steps. The first step is to determine the corresponding block in the reference frame using the time vector. The reference image is called the motion source image. The second step is to divide the current CU into sub-CUs and obtain the motion vector and reference index of each sub-CU from the block corresponding to the sub-CU.

In the first step, the reference image and the corresponding block are determined based on the motion information of the spatial neighboring block of the current CU. To avoid a repetitive scanning process for a neighboring block, the first merge candidate in the list of merge candidates for the current CU is used. The first available motion vector and the reference index associated with the first available motion vector are set as the temporal vector and index of the original motion image. Thus, compared to TMVP, in ATMVP the corresponding block can be identified more accurately. In TMVP, the corresponding box (sometimes called a merged box) is always located in the lower right corner or centered on the current CU.

In the second step, the corresponding sub-CU is identified by a time vector in the motion source image by adding the time vector to the coordinates of the current CU. For each sub-CU, motion information of the corresponding block (the smallest motion grid covering the center sample) of the sub-CU is used to obtain motion information for the sub-CU. After the motion information of the corresponding N×N block is identified, in HEVC, the motion information is converted into a motion vector and a reference index of the current sub-CU in the same manner as TMVP, and motion scaling and other procedures are applied. For example, the decoder checks whether a low delay condition is satisfied (that is, the POC of all reference pictures of the current picture is less than the POC of the current picture), and possibly uses a motion vector MVx (a motion vector corresponding to the reference picture list X) to predict the motion vector MVy (with X equal to 0 or 1 and Y equal to 1 - X) for each sub-CU.

Spatiotemporal Motion Vector Prediction

In this method, the motion vector of the sub-CU is obtained recursively following the raster scan order. An 8×8 CU is considered to include four 4×4 sub-CUs A, B, C, and D. Adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

Getting motion for sub-CU A begins with the identification of two spatial neighbors of sub-CU A. The first neighbor is an N×N block over sub-CU A (namely, block c). If block c is not available or intra-coded, other N × N blocks above sub-CU A are checked (from left to right, starting from block c). The second neighbor is the block to the left of sub-CU A (namely, block b). If block b is not available or intra-coded, other blocks to the left of sub-CU A are checked (from top to bottom, starting from block b). The motion information obtained from the neighboring block for each list is changed in accordance with the first reference frame for that list. The temporal motion vector prediction value (TMVP) of sub-block A is then obtained by following the same TMVP acquisition procedure as specified in HEVC. The motion information of the co-located block at position D is selected and scaled accordingly. Finally, after receiving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The average motion vector is assigned as the motion vector of the current sub-CU.

Merged with merge mode

The sub-CU mode is used as an additional merge candidate, and no additional syntax element is required for mode signaling. Two additional merge candidates are added to the merge candidate list for each CU to represent the ATMVP mode and the STMVP mode. If the sequence parameter set specifies the use of ATMVP and STMVP, up to seven merge candidates are used. The logic for encoding additional merge candidates is the same as the logic for encoding a merge candidate in HM. This means that for each CU in the P- or B-segment, two more RD checks must be performed for two additional merge candidates.

Affine Motion Compensation Prediction

The block's affine motion field is described using two control point motion vectors.

The motion vector field (MVF) of the block is described according to the following equation:

(1.6)

(1.6)

where (v0x, v0y) represents the motion vector of the control point in the upper left corner and (v1x, v1y) represents the motion vector of the control point in the upper right corner.

To further simplify motion compensation prediction, sub-block based affine transform prediction is applied. The sub-block size M x N is derived according to equation (1.7), where MvPre represents the fractional motion vector precision (e.g., 1/16), and (v2x, v2y) represents the motion vector of the lower left control point calculated according to equation (1.6).

(1.7)

(1.7)

After the subblock size M × N is obtained according to equation (1.7), M and N must be adjusted downwards, if necessary, to make M and N divisors of w and h, respectively.

To obtain the motion vector of each sub-block M × N, the motion vector of the central sample of the sub-block is calculated according to equation (1.6) and rounded to fractional precision 1/16.

Affine outer regime

создается с использованием соседнего блока.

выбирается из вектора движения блока A, B или C. Вектор движения из соседнего блока масштабируется в соответствии со справочном списком и соотношением между POC ссылки для соседнего блока, POC ссылки для текущего CU и POC текущего CU. Подход, используемый для выбора

из соседних блоков D и E, аналогичен. Если количество списков кандидатов меньше 2, список дополняется парой векторов движения, составленной путем дублирования каждого кандидата AMVP. Когда количество списков кандидатов больше 2, кандидаты сначала сортируются в соответствии с согласованностью соседних векторов движения (сходство двух векторов движения в паре кандидатов), и сохраняются только первые два кандидата. Проверка стоимости RD используется для определения, какой кандидат пары векторов движения выбран в качестве предсказания вектора движения контрольной точки (control point motion vector prediction, CPMVP) текущего CU, и указывается индекс, указывающий позицию CPMVP в списке кандидатов в битовом потоке. Разница между CPMV и CPMVP сообщается в битовым потоке.For a CU whose width and height is greater than 8, the AF_INTER mode can be applied. An affinity flag at the CU level is signaled in the bitstream to indicate whether the AF_INTER mode is used. In this mode, the list of candidates with a pair of motion vectors

created using an adjacent block.

is selected from the motion vector of block A, B, or C. The motion vector from the neighboring block is scaled according to the look-up list and the relationship between the link POC for the neighbor block, the link POC for the current CU, and the POC of the current CU. Approach used for selection

from neighboring blocks D and E is similar. If the number of candidate lists is less than 2, the list is padded with a pair of motion vectors composed by duplicating each AMVP candidate. When the number of candidate lists is greater than 2, the candidates are first sorted according to adjacent motion vector consistency (similarity of two motion vectors in a candidate pair), and only the first two candidates are kept. The RD cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) of the current CU, and an index indicating the position of the CPMVP in the candidate list in the bitstream is indicated. The difference between CPMV and CPMVP is reported in the bitstream.

Affine merge mode

,

и

движения, и CU, включающий в себя блок A. Дополнительно, вектор

движения в верхнем левом углу текущего CU вычисляется согласно

,

и

. Затем вычисляется правый верхний вектор

движения текущего CU.When the CU is applied in the AF_MERGE mode, the first block encoded based on the affine mode is obtained from the actual neighboring reconstructed block. The block candidate selection order is left, top, top right, bottom left, top left. If the bottom-left neighbor block A is encoded based on the affine mode, then top-left, top-right, and bottom-left vectors are output.

,

and

movement, and a CU including block A. Additionally, the vector

movement in the upper left corner of the current CU is calculated according to

,

and

. Then the upper right vector is calculated

movement of the current CU.

To identify whether the current CU is encoded based on the AF_MERGE mode, when there is at least one neighboring block that is encoded based on the affine mode, an affine flag is signaled in the bitstream.

Getting a motion vector with a matched pattern

The pattern matched motion vector derivation (PMMVD) mode is based on Frame-Rate up Conversion (FRUC) technologies. In this mode, block motion information is not transmitted, but received at the decoder side.

When a CU's merge flag is true, that CU's FRUC flag is signaled. When the FRUC flag is false, the merge index is specified and normal merge mode is used. When the FRUC flag is true, an additional FRUC mode flag is signaled to indicate which method (two-way matching or pattern matching) should be used to obtain block motion information.

On the encoder side, the decision whether to use the FRUC merge mode for the CU is based on the choice of RD cost, as is done for a normal merge candidate. That is, two matching modes (two-way matching and pattern matching) are tested for the CU with the choice of RD cost. The mode leading to the minimum cost is further compared with other CU modes. If the FRUC mapping mode is the most efficient mode, the FRUC flag is set to true for the CU and the associated mapping mode is used.

The process of obtaining traffic in FRUC merge mode includes two steps. First, a motion search is performed at the CU level, followed by motion refinement at the CU sublevel. At the CU level, an initial motion vector for the entire CU is obtained based on two-way matching or pattern matching. First, a list of MV candidates is compiled and the candidate that results in the lowest matching cost is selected as a starting point for further refinement at the CU level. A local search is then performed based on a two-way or pattern matching around the starting point, and the MV that results in the lowest matching cost is used as the MV for the entire CU. Subsequently, the motion information is further refined at the sub-CU level using the received motion vector of the CU as a starting point.

For example, the following acquisition process is performed to output W x H CU motion information. At the first stage, the MV of the total W x H CU is displayed. In the second step, the CU is further divided into M x M sub-CUs. The value of M is calculated in accordance with Equation (1.8), D represents a predetermined separation depth and defaulted to 3 in the JEM. The MV of each sub-CU is then obtained.

(1.8)

(1.8)

Two-way matching is used to obtain motion information of the current CU by finding the closest match between two blocks along the motion path of the current CU in two different reference pictures. Assuming a continuous motion path, the motion vectors MV0 and MV1 pointing to two reference blocks should be proportional to the time distances, ie, TD0 and TD1, between the current picture and the two reference pictures. When the current picture is temporal between two reference pictures and the temporal distances between the current picture and the two reference pictures are the same, the bi-directional matching becomes a mirror-based bi-directional MV.

In the two-way matching merge mode, bi-directional prediction is always applied because CU motion information is obtained based on the closest match between two blocks along the motion path of the current CU in two different reference pictures. For pattern-matching merge mode, there is no such restriction. In the pattern-matching merge mode, the encoder may select unified prediction from list 0, unified prediction from list 1, or dual prediction for the CU. The selection is based on the following pattern matching cost:

If costBi <= factor * min(cost0, cost1),

bidirectional prediction is used;

Otherwise, if cost0 <= cost1,

the unified prediction from list 0 is used;

Otherwise,

the unified prediction from list 1 is used.

cost0 is the list-0 pattern matching SAD, cost1 is the list-1 pattern matching SAD, and costBi is the double prediction pattern matching SAD. The coefficient value is 1.25, which means that the selection process is biased towards two-dimensional prediction. The inter prediction direction selection applies only to the pattern matching process at the CU level.

Pattern matching is used to obtain motion information of the current CU by finding the closest match between a pattern (top and/or left adjacent blocks of the current CU) in the current picture and a block (with the same size as the pattern) in the reference picture. With the exception of the aforementioned FRUC merge mode, pattern matching also applies to the AMVP mode. A new candidate is obtained using a pattern matching method. If the new candidate obtained by pattern matching is different from the first existing AMVP candidate, the new candidate is inserted at the very beginning of the AMVP candidate list and then the list size is set to 2 (meaning the second existing AMVP candidate is deleted). When pattern matching is applied to AMVP mode, only CU-level lookup is applied.

MV candidates established at the CU level include: a. the initial AMVP candidate selected if the AMVP mode is used for the current CU; b. all merger candidates; c. multiple MVs in an interpolated MV field; and d. top and left adjacent motion vectors.

It should be noted that the interpolated MV field mentioned above is generated before encoding the entire picture based on the one-way ME. The motion field may then be used later as an MV candidate at the CU level or sub-CU level. First, the motion field of each reference picture in the two reference lists intersects at the 4×4 block level. For each 4×4 block, if the motion associated with the block passes through 4×4 blocks in the current image and the block has not been assigned any interpolated motion, the motion of the reference block is scaled to the current image based on the temporal distances TD 0 and TD 1 (similar to MV scaling in TMVP in HEVC) and the scaled motion is assigned to a block in the current frame. If a scaled MV is not assigned to a 4×4 block, the block's motion in the interpolated motion field is marked as unavailable.

When two-way matching is used, each valid merge candidate MV is used as input to generate a pair of MVs with a two-way matching assumption. For example, one valid merge candidate MV is (MVa, refa) in reference list A. Then the reference image refb of its paired two-sided MV is in another reference list B. Thus, refa and refb are temporarily located on opposite sides of the current image. If no such refb is available in link list B, refb is defined as a reference other than refa, and the temporal distance between refb and current image is the minimum in list B. Once refb is determined, MVb is obtained by scaling MVa based on temporal distances between current image and refa and between current image and refb.

The four MVs from the interpolated MV field are also added to the candidate list at the CU level. More specifically, interpolated MVs are added at positions (0, 0), (W/2, 0), (0, H/2) and (W/2, H/2) of the current CU.

When FRUC is applied to the AMVP mode, the original AMVP candidate is also added to the CU layer MV candidate set.

At the CU level, up to 15 AMVP CU MVs and up to 13 merged CU MVs are added to the candidate list.

MV candidates established at the sub-CU level include: a. MV determined from CU level search; b. top, left, top left, and top right neighbor MVs; c. a scaled version of the combined MV from the reference image; d. up to four ATMVP candidates; and e. up to four STMVP candidates.

The scaled MV from the reference picture is obtained as follows: all reference pictures in both lists are moved, and the MV at the aligned sub-CU position in the reference picture is scaled to the reference original MV of the CU level.

ATMVP and STMVP candidates are limited to the first four.

At the CU sublevel, up to 17 MVs are added to the list of candidates.

Refinement of the motion vector

The motion vector can be refined using various methods combined with different inter prediction modes.

Refinement of MV in FRUC

MV refinement is a search for MV based on a template with a two-way matching cost or template matching cost criterion. Two search patterns are supported in current development: unrestricted center-biased diamond search (UCBDS) and adaptive cross search used to refine MVs at the CU and sub-CU levels, respectively. For MV refinement at both the CU level and the CU sublayer level, the MV is searched directly with the MV accuracy of a quarter luminance sample, after which the MV refinement of one eighth luma sample is performed. The MV refinement search range for the CU and the sub-CU stage is set to 8 luma samples.

Motion vector refinement on the decoder side

In the dual prediction operation for predicting one area block, two prediction blocks generated using List-0 MV and List-1 MV, respectively, are combined to generate a single prediction signal. In a decoder-side motion vector refinement (DMVR) method, two bidirectional prediction motion vectors are further refined using a two-way pattern matching process. Two-sided pattern matching is applied in the decoder to search for distortions between the two-sided pattern and the reconstructed sample in the reference picture, and to obtain an improved MV without sending additional motion information.

In DMVR, a bi-directional pattern is generated as a weighted combination (namely, the average) of two prediction blocks from the initial MV 0 list-0 and MV 1 list-1, respectively. The pattern matching operation includes calculating a cost value between the generated pattern and a region sample (around the prediction start block) in the reference picture. For each of the two reference pictures, the MV that yields the minimum template cost is treated as the updated MV for the list to replace the original MV. In the current version, nine candidate MVs are searched for each list. The nine candidate MVs include the original MV and eight surrounding MVs with one luma sample offset from the original MV in either the horizontal direction, the vertical direction, or the horizontal and vertical directions. Finally, two new MVs, ie, MV 0' and MV 1' are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as a cost measure.

DMVR is applied for bidirectional prediction fusion mode with one MV from a reference picture in the past and another MV from a reference picture in the future, without sending additional syntax elements.

Movement Accuracy and Storage

Reducing Motion Data Storage

The use of TMVP in AMVP as well as in fusion mode requires the storage of motion data (including motion vector, reference index, and coding mode) in the merged reference picture. Considering the granularity of motion representation, the amount of memory required to store motion data is significant. HEVC uses motion data storage reduction (MDSR) to reduce the size of the motion data buffer and its associated memory access bandwidth over subsample motion data in a reference frame. Although information is stored as 4×4 blocks in H.264/AVC, HEVC uses a 16×16 block. In the case of 4×4 grid subsampling, 4×4 upper left block information is stored. Due to this subsampling, MDSR affects the quality of the temporal predictions.

In addition, there is a close correlation between the position of the MV used in the combined image and the position of the MV stored using MDSR. During the HEVC standardization process, it appears that keeping the top left block motion data inside a 16×16 region along with the bottom right and center TMVP candidates provides the best trade-off between coding efficiency and memory bandwidth reduction.

Higher motion vector accuracy

In HEVC, motion vector accuracy is one quarter of a pixel (one fourth luma sample and one eighth chroma sample for 4:2:0 video). In current developments, the accuracy of internal storage of motion vectors and fusion candidates is increased to 1/16th of a pixel. Higher motion vector precision (1/16 pixel) is used in motion compensation inter prediction for CU coded based on the skip/merge mode. For a CU encoded in normal AMVP mode, integer pixel motion or quarter pixel motion is used.

Adaptive Motion Vector Difference Resolution

In HEVC, the motion vector difference (MVD) is reported in units of a luma sample quarter when use_integer_mv_flag is 0 in the segment header. The current version introduces locally adaptive motion vector resolution (LAMVR). The MVD may be encoded in units of a fourth luma sample, an integer luminance sample, or four luma samples. MVD permission is controlled at the coding unit (CU) level, and an MVD permission flag is conditionally signaled for each CU that has at least one non-zero MVD component.

For a CU that has at least one non-zero MVD component, a first flag is signaled to indicate whether the quarter luma sample precision MV for the CU is used. When the first flag (which is 1) indicates that the quarter luminance sample precision MV is not used, another flag is signaled to indicate whether the integer luminance sample precision MV or the four luminance sample precision MV is used.

When the first CU MVD resolution flag is zero or not encoded for the CU (meaning that all MVDs for the CU are zero), a quarter of the luma sample resolution MV is used for the CU. When integer luma sample precision MV or CU four luminance sample precision MV is used, the MVP in the AMVP candidate list for the CU is rounded to the appropriate precision.

The encoder uses a CU-level RD check to determine which MVD resolution should be used for the CU. That is, the RD check at the CU level is performed three times for each MVD permission.

Fractional interpolation module

When the motion vector points to a fractional sample position, motion-compensated interpolation is required. For luminance interpolation filtering, an 8-tap DCT-based separable interpolation filter is used for 2/4 sampling, and a 7-tap DCT-based split interpolation filter is used for 1/4 sampling, as shown in Table 1.2.

Table 1.2

Position Filter Coefficients 1/4 {-1, 4, -10, 58, 17, -5, 1} 2/4 {-1, 4, -11, 40, 40, -11, 4, -1} 3/4 {1, -5, 17, 58, -10, 4, -1}

Similarly, a 4-tap DCT-based interpolation filter is used for the chrominance interpolation filter, as shown in Table 1.3.

Table 1.3.

Position Filter Coefficients 1/8 {-2, 58, 10, -2} 2/8 {-4, 54, 16, -2} 3/8 {-6, 46, 28, -4} 4/8 {–4, 36, 36, –4} 5/8 {-4, 28, 46, -6} 6/8 {-2, 16, 54, -4} 7/8 {-2, 10, 58, -2}

For vertical interpolation for 4:2:2 and horizontal and vertical interpolation for 4:4:4 chroma channels, the odd positions in Table 1.3 are not used, resulting in 1/4th chroma interpolation.

For bidirectional prediction, the bit depth of the output of the interpolation filter is maintained to within 14 bits, regardless of the original bit depth, before averaging the two prediction signals. The actual averaging process is done implicitly with the bit depth reduction process as follows:

(1.9)

(1.9)

(1.10)

(1.10)

(1.11)

(1.11)

To reduce complexity, bilinear interpolation is used for bi-directional and pattern matching instead of the usual 8-tap HEVC interpolation.

The calculation of the cost of matching at different stages is slightly different. When a candidate is selected from a set of candidates at the CU level, the matching cost is the SAD of the two-way or pattern matching. Once the initial MV is determined, the two-way matching matching cost C in the CU sublayer search is calculated as follows:

(1.12)

(1.12)

и

указывают текущее MV и начальное MV, соответственно. SAD по-прежнему используется в качестве стоимости сопоставления шаблонов при поиске на уровне суб-CU.w is a weighting factor that is empirically set to 4, and

and

indicate the current MV and the initial MV, respectively. SAD is still used as the cost of pattern matching in sub-CU searches.

In FRUC mode, MV is output only using luminance sampling. The resulting motion vector should be used for both luminance and chrominance for MC inter prediction. After MV is determined, the final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chrominance.

Motion Compensation Module

Overlapping Block Motion Compensation

Overlapped Block Motion Compensation (OBMC) is performed for all motion compensation (MC) block boundaries except for the right and bottom CU boundaries in the current version. In addition, overlapping block motion compensation is applied to both luminance and chrominance components. The MC block corresponds to the coding block. When a CU is encoded based on a sub-CU mode (including sub-CU merge, affine, and FRUC modes), each sub-CU is an MC. In order to handle the CU boundary in a uniform manner, OBMC is performed at the sub-unit level for all MC-unit boundaries, where the sub-unit size is set to 4×4.

When OBMC is applied to the current subblock, in addition to the current motion vector, if the motion vectors of the four associated neighboring subblocks are available and are not identical to the current motion vector, the motion vectors of the four associated neighboring subblocks are also used to obtain a prediction block for the current subblock. The plurality of these prediction blocks based on the plurality of motion vectors are combined to generate the final prediction signal of the current sub-block.

A prediction block based on the motion vector of an adjacent sub-block is referred to as PN, where N represents an index of the top, bottom, left, or right neighboring sub-block, and a prediction block based on the motion vector of the current sub-block is referred to as PC. When the PN is based on the motion information of a neighboring subblock that includes the same motion information as the current subblock, OBMC is not executed from the PN. Otherwise, each PN sample is added to the same sample in the PC. That is, four rows/columns PN are added to the PC. Weights {1/4, 1/8, 1/16, 1/32} are used for PN and weights {3/4, 7/8, 15/16, 31/32} are used for PC. An exception is a small MC block (ie, when the height or width of the coding block is 4 or the CU is encoded based on the sub-CU mode), and only two PN rows/columns are added to the PC for such a block. In this case, weights {1/4, 1/8} are used for PN and weights {3/4, 7/8} are used for PC. For a PN generated based on the motion vector of a vertical (horizontal) neighboring subblock, samples in the same row (column) of the PN are added to the PC with the same weight.

In current versions, for CUs with a size less than or equal to 256 luma samples, a CU level flag is signaled to indicate whether OBMC is applied to the current CU. For a CU larger than 256 luminance samples or a CU not encoded based on the AMVP mode, OBMC is applied by default. In the encoder, when OBMC is applied to a CU, the effect of OBMC is taken into account in the motion estimation step. The prediction signal generated by OBMC using the motion information of the upper neighbor block and the left neighbor block is used to compensate the upper and left boundaries of the original signal of the current CU, and then normal motion estimation processing is applied.

Optimization tools

Local Light Compensation

Local Illumination Compensation (LIC) is based on a linear model for illumination changes using a scaling factor a and an offset b. Local illumination compensation is turned on or off adaptively for each coding unit (CU).

When LIC is applied to a CU, a least squares error method is used to derive parameters a and b using the current CU's neighbor sample and the corresponding reference sample. The sub-samples (2:1 sub-samples) of the adjacent CU sample and the corresponding sample (identified by the motion information of the current CU or sub-CU) are used as the reference picture. The LIC parameters are derived and applied for each prediction direction separately.

When a CU is encoded based on a merge mode, the LIC flag is copied from an adjacent block in a manner similar to copying motion information in a merge mode. Otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applied or not.

When LIC is used for an image, an additional RD check is needed at the CU level to determine if LIC is applied to the CU. When LIC is used for CU, the mean-removed sum of absolute difference (MR-SAD) and the mean-removed sum of absolute Hadamard-transformed difference (MR -SATD) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively.

Bidirectional optical flow

Bi-directional Optical flow (BIO) is a discrete motion refinement performed on top of block motion compensation for bidirectional prediction. Discrete motion refinement does not use signaling.

является значением яркости из опорной к (где к = 0, 1) после блока компенсации движения, и

и

представляют собой горизонтальную составляющую и вертикальную составляющую

градиента, соответственно. Предполагая, что оптический поток действителен, поле

вектора движения задается в соответствии с уравнением (1.13):Let be

is the brightness value from the reference k (where k = 0, 1) after the motion compensation block, and

and

are the horizontal component and the vertical component

gradient, respectively. Assuming the optical flow is valid, the field

motion vector is given in accordance with equation (1.13):

(1.13)

(1.13)

функции, так и производным

и

на концах. Значение этого полинома при t = 0 является значением предсказания BIO:Combining this optical flow equation with Hermite interpolation for the motion path of each sample results in a unique third-order polynomial that corresponds to both the value

functions and derivatives

and

at the ends. The value of this polynomial at t = 0 is the value of the BIO prediction:

(1.14)

(1.14)

и

обозначают расстояния до опорного кадра. Расстояния

и

рассчитываются на основании POCs для Ref0 и Ref1:

= POC (текущий) - POC (Ref0),

= POC (Ref1) - POC (текущий). Если оба предсказания исходят из одного и того же временного направления (либо из прошлого, либо из будущего), знаки разные (то есть,

). В этом случае BIO применяется только в том случае, если предсказания не относятся к одному и тому же моменту времени (то есть,

), обе упомянутые области имеют ненулевые векторы (

) движения и векторы движения блока пропорциональны временным расстояниям In this description

and

denote the distances to the reference frame. Distances

and

are calculated based on POCs for Ref0 and Ref1:

= POC(current) - POC(Ref0),

=POC(Ref1) - POC(current). If both predictions come from the same time direction (either past or future), the signs are different (i.e.,

). In this case, BIO only applies if the predictions do not refer to the same point in time (i.e.,

), both mentioned regions have nonzero vectors (

) motions and motion vectors of the block are proportional to the time distances

).(

).

вектора движения определяется путем минимизации разности Δ между значениями в точках А и В (пересечение траектории движения и плоскостей опорных кадров). Модель использует только первый линейный член локального разложения Тейлора для ∆:Field

the motion vector is determined by minimizing the difference Δ between the values at points A and B (the intersection of the motion trajectory and reference frame planes). The model uses only the first linear term of the local Taylor expansion for ∆:

(1.15)

(1.15)

выборки, которое до сих пор не указывалось в обозначениях. Предполагая, что движение согласовано в локальной окружающей области, ∆ минимизируется внутри (2M + 1) х (2M + 1) квадратного окна

с центром в текущей предсказанной точке

, где M равно 2:All values in equation (1.15) depend on the location

sample, which has not yet been indicated in the notation. Assuming motion is consistent in the local surrounding area, ∆ is minimized within the (2M + 1) x (2M + 1) square window

centered on the current predicted point

, where M is equal to 2:

(1.16)

(1.16)

For this optimization problem, the current version uses a simplified approach that performs minimization first in the vertical direction and then in the horizontal direction. It turns out the following:

(1.17)

(1.17)

(1.18)

(1.18)

where

(1.19)

(1.19)

To avoid division by zero or a very small value, the regularization parameters r and m are used in equations (1.17) and (1.18).

(1.20)

(1.20)

(1.21)

(1.21)

Here d represents the bit depth of the video sample.

градиента только для позиций внутри текущего блока. В уравнении (1.19) (2M + 1) х (2M + 1) квадратное окно Ω с центром в текущей точке предсказания на границе блока предсказания должно иметь доступ к позиции за пределами блока. В текущей версии значения

вне блока устанавливаются равными ближайшему доступному значению внутри блока. Например, это можно реализовать как заполнение.To ensure that the memory access for BIO remains the same as the memory access for conventional bidirectional motion compensation, all predictions and values are computed.

gradient only for positions inside the current block. In equation (1.19) (2M + 1) x (2M + 1) a square window Ω centered on the current prediction point on the boundary of the prediction block must access a position outside the block. In the current version of the value

outside the block are set to the nearest available value inside the block. For example, this can be implemented as padding.

With BIO it is possible that the motion field is refined for each sample. However, the BIO block design can be used to reduce computational complexity. The motion refinement is computed based on the 4×4 block. In the block BIO, the sn values in equation (1.19) of all samples in the 4×4 block are aggregated, and then the aggregated sn values are used to obtain the BIO motion vector offset for the 4×4 block. The following formula is used to getting BIO based on blocks:

(1,22)

(1.22)

bk denotes the set of samples belonging to the k-th 4×4 block of the prediction block. sn in equations (1.17) and (1.18) is replaced by ((sn, bk) >> 4) to obtain the corresponding motion vector offset.

; в противном случае, устанавливается значение

. In some cases, the MV group BIO may be unreliable due to noise or uneven movement. Therefore, in BIO, the MV value of the group is limited by the threshold value thBIO. The threshold value is determined based on whether all reference pictures of the current picture are in the same direction. If all reference pictures of the current picture come from the same direction, the threshold value is set to

; otherwise, set to

.

сначала применяется фильтр градиента по вертикали с использованием BIOfilterG, соответствующего дробной позиции fracY со сдвигом масштабирования d - 8, и затем смещение сигнала выполняется с помощью BIOfilterS в горизонтальном направлении, соответствующем дробной позиции fracX со сдвигом уменьшения масштаба на 18-d. Длины интерполяционных фильтров для вычисления BIOfilterG градиента и смещения BIOfilterS сигнала короче (6 отводов) для сохранения надлежащей сложности. В таблице 1.4 показан фильтр, используемый для вычисления градиента в различных дробных позициях вектора движения блока в BIO. В таблице 1.5 показан фильтр интерполяции для генерирования сигнала предсказания в BIO.The gradient for BIO is calculated simultaneously with motion compensation interpolation using an operation corresponding to the HEVC (two-dimensional shared FIR) motion compensation process. The 2D input of the shared FIR is the same reference frame sample as the motion compensation process and the fractional positions (fracX, fracY) according to the fractional part of the block's motion vector. In the case of a horizontal gradient signal, a vertical interpolation is first performed using BIOfilterS corresponding to the fractional position of fracY with a scaling offset of d-8, and then applying a gradient filter BIOfilterG in the horizontal direction corresponding to the fractional position of fracX with a scaling offset of 18-d. In the case of a vertical gradient

first a vertical gradient filter is applied using BIOfilterG corresponding to the fractional position of fracY with a scaling offset of d - 8, and then the signal is shifted using BIOfilterS in the horizontal direction corresponding to the fractional position of fracX with a downscaling offset of 18-d. The interpolation filter lengths for calculating the gradient BIOfilterG and the signal offset BIOfilterS are shorter (6 taps) to keep proper complexity. Table 1.4 shows the filter used to calculate the gradient at various fractional positions of the block's motion vector in the BIO. Table 1.5 shows the interpolation filter for generating the prediction signal in BIO.

Table 1.4

Fractional pixel position Gradient interpolation filter (BIOfilterG) 0 {8, -39, -3, 46, -17, 5} 1/16 {8, -32, -13, 50, -18, 5} 1/8 {7, -27, -20, 54, -19, 5} 3/16 {6, -21, -29, 57, -18, 5} 1/4 {4, -17, -36, 60, -15, 4} 5/16 {3, -9, -44, 61, -15, 4} 3/8 {1, -4, -48, 61, -13, 3} 7/16 {0, 1, -54, 60, -9, 2} 1/2 {-1, 4, -57, 57, -4, 1}

Table 1.5

Fractional pixel position Interpolation filter for prediction signal (BIOfilterS) 0 {0, 0, 64, 0, 0, 0} 1/16 {1, -3, 64, 4, -2, 0} 1/8 {1, -6, 62, 9, -3, 1} 3/16 {2, -8, 60, 14, -5, 1} 1/4 {2, -9, 57, 19, -7, 2} 5/16 {3, -10, 53, 24, -8, 2} 3/8 {3, -11, 50, 29, -9, 2} 7/16 {3, -11, 44, 35, -10, 3} 1/2 {3, -10, 35, 44, -11, 3}

In the current version, BIO is applied to all bidirectional prediction blocks when two predictions are made from different reference pictures. When LIC is used for CU, BIO is disabled. OBMC is applied to the block after the normal MC process. To reduce computational complexity, BIO is not applied in the OBMC process. This means that the BIO is only applied in the MC process for a block when the block MV is used, but not applied in the MC process when the neighboring block MV is used in the OBMC process.

Weighted Sample Predictor

A weighted prediction (WP) tool is provided as a possible tool in HEVC. The principle of WP is to replace the inter prediction signal P with a linear weighted prediction signal P' (with weight w and bias o):

unified prediction: P'= w × P + o (1.23)

bidirectional prediction: P'= (w0 × P0 + o0 + w1 × P1 + o1) / 2 (1.24)

The applicable weight and applicable offset are selected by the encoder and transmitted in the bitstream. The suffixes L0 and L1 define list 0 and list 1 of reference picture lists, respectively. For the interpolation filter, the bit depth is maintained to 14 bits before averaging the prediction signals.

In the case of bi-directional prediction with at least one reference picture available in each of the lists L0 and L1, the following formula is applied to the explicit signaling of the weighted prediction parameter related to the luminance channel. The corresponding formula is applied to the chrominance channel and to the case of unified prediction.

(1.25)

where

,

,

,

,

,

,

,

,

,

,

.

.

The Boundary prediction filters are an intra-coding technique used to further customize the prediction samples in the leftmost column and topmost row. In HEVC, after an intra prediction block has been generated for vertical or horizontal intra mode, the prediction samples in the leftmost column and topmost row are further adjusted accordingly. This method can be further extended to multiple diagonal internal modes, and border samples up to four columns or rows are further configured with a 2-tap filter (for internal modes 2 and 34) or a 3-tap filter (for internal modes 3 to 6 and 30). -33).

In the HEVC standard and prior standards, reference frames are classified into two groups: a forward reference frame and a reverse reference frame, and are respectively placed in two reference frame lists (reference picture list). The two reference frame lists are commonly referred to as list 0 and list 1. For the current block, the inter prediction direction is used to indicate the prediction direction in forward prediction, backward prediction, or bidirectional prediction, and the other reference frame list, i.e., list 0, list 1, or list 0 and list 1 are selected based on the prediction direction. The key frame in the selected selection list is indicated by the key frame index. The motion vector is used to indicate the offset position of the prediction reference block of the current block in the selected sampling system, relative to the current block in the current frame. Then, the final prediction block is generated based on the prediction direction using block prediction obtained from the reference frame in list 0, list 1, or both lists 0 and 1. When the prediction direction is unified prediction, the prediction block obtained from the reference frame in list is directly used. 0 or 1. When the prediction direction is bidirectional prediction, prediction blocks obtained from reference frames in both list 0 and list 1 are synthesized by weighted averaging to obtain a final prediction block.

In order to solve the technical problem in the conventional technology that the prediction samples obtained based on the inter prediction mode are spatially discontinuous, which affects the prediction performance, and the prediction residual energy is relatively high, embodiments of the present invention provide an inter prediction method for filtering the prediction sample using an adjacent reconstructed sample after the prediction sample is generated, which improves coding efficiency.

Fig. 13 is a flowchart of an inter prediction method according to an embodiment of the present invention. As shown in FIG. 13, the method includes steps S1301 to S1307.

S1301: Bitstream parsing executor to obtain motion information of the image block to be processed.

The image block being processed may be referred to as the current block or the current CU.

It can be understood that step S1301 may be performed by video decoder 200 in FIG. one.

For example, in this embodiment of the present invention, a block-based motion compensation technique can be used to search for encoded blocks for the best match block of the current coding block, so that the residual between the prediction block and the current block is as small as possible; and is used to calculate the MV offset of the current block.

For example, the image block to be processed can be any image block, and the size of the image block to be processed can be 2 x 2, 4 x 4, 8 x 8, 16 x 16, 32 x 32, 64 x 64, or 128 x 128 This is not limited in this embodiment of the present invention.

For example, if the image block to be processed (the current block) is encoded at the encoder side based on the merge mode, the spatial candidate and temporal candidate of the current block may be added to the candidate list with merge motion information for the current block. The method is the same as the method in HEVC. For example, any of the technologies described in FIG. 8-fig. 12 may be used as a specific method for obtaining a candidate list with merge motion information.

For example, if the merge mode is used for the current block, the motion information of the current block is determined based on the merge index carried in the bitstream. If the inter-MVP mode for the current block is used, the motion information of the current block is determined based on the inter-prediction direction, the reference frame index, the motion prediction vector index value, and the residual motion vector cost, which are transmitted in the bitstream.

Step S1301 may be performed using a method in HEVC or VTM, or may be performed using another method to list motion vector prediction candidates. This is not limited to this embodiment of the present invention.

S1302: Determine to update the prediction block of the image block to be processed (may be) determined.

It can be understood that step S1302 may be performed by video decoder 200 in FIG. one.

The prediction block of the image block to be processed is the prediction block of the current block, and may be obtained based on one or more coded blocks.

For example, whether it is necessary to update the prediction block of the image block to be processed may be determined based on the update determination indicator of the image block to be processed. In other words, whether to perform spatial filtering of the image block to be processed can be determined based on the update determination indicator of the image block to be processed.

In an exemplary implementation, the bitstream may be parsed to obtain update definition indication information of the image block to be processed, where the update definition indication information is used to indicate whether to update the prediction block of the image block to be processed; and further determined, based on the update determination indication information of the image block to be processed, that the prediction block of the image block to be processed is to be updated.

In another exemplary implementation, predetermined update determination indication information of the image block to be processed can be obtained, where the preset update determination indication information is used to indicate whether to update the prediction block of the image block to be processed; and further determined based on the given update determination indication information that the prediction block of the image block to be processed is to be updated.

For example, if the update determination indicator is true, it may be determined that the prediction block of the image block to be processed is to be updated. In other words, it is determined that spatial filtering is to be performed on a prediction block of an image block to be processed. If the update determination indicator is false, it is determined that the prediction block of the image block to be processed does not need to be updated. The specific form of the update determination indicator is not limited in this embodiment of the present invention. Here, whether the update determination indicator is true or false is simply used as an example for description.

S1303: (Could) determine the prediction mode corresponding to the image block to be processed.

It can be understood that step S1303 may be performed by video decoder 200 in FIG. one.

For example, the prediction mode corresponding to the image block to be processed may be a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode. This is not limited to this embodiment of the present invention. It can be understood that the prediction mode corresponding to a block of an image to be processed may be a fusion only mode, an inter-AMVP only mode, or a combination of a fusion mode and an inter-AMVP mode.

It should be noted that the advanced inter-motion vector prediction (inter-AMVP) mode may also be referred to as the inter-motion vector prediction (inter-MVP) mode.

For example, the method for determining the prediction mode corresponding to the image block to be processed may be as follows: parsing the bitstream to obtain the prediction mode corresponding to the image block to be processed, and determining that the prediction mode corresponding to the image block to be processed is a fusion mode. and/or inter-AMVP mode.

It can be understood that in this embodiment of the present invention, the spatial filtering method can be performed on an inter-coded block that has been coded based on a merge mode and/or an inter-AMVP mode. In other words, filtering processing may be performed at the decoder side during decoding for a block that has been encoded based on the merge mode and/or the inter-AMVP mode.

S1304: perform motion compensation on the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed.

The prediction block of the image to be processed includes the prediction value of the target sample.

It can be understood that step S1304 may be performed by video decoder 200 in FIG. one.

So, for example, during motion compensation, the current partial image is predicted and compensated based on the reference image. This can reduce the redundancy of information in a sequence of frames.

For example, when motion compensation is performed based on motion information, a prediction block of an image block to be processed can be obtained from a reference frame based on the direction of the reference frame, the sequence number of the reference frame, and the motion vector. The direction of the reference frame may be forward prediction, reverse prediction, or bidirectional prediction. This is not limited to this embodiment of the present invention.

For example, when the fetch direction is direct prediction, a reference frame may be selected from the front reference picture set to encode the current block (CU) to obtain the reference block. When the fetch direction is inverse prediction, the reference picture may be selected from the inverse reference picture set for the current coding unit (CU) to obtain the reference block. When the sampling direction is bidirectional prediction, a reference picture may be selected from each of the forward reference picture set and the reverse reference picture set for the current coding unit (CU) to obtain the reference block.

It should be noted that in step S1304, the method for performing motion compensation on the image block to be processed based on the motion information may be a method in HEVC or VTM, or may be another method for obtaining a prediction block of the image block to be processed. This is not limited to this embodiment of the present invention.

S1306: Perform a weighting calculation of one or more recovered values of one or more reference samples and a prediction value of the target sample to update the prediction value of the target sample.

The reference sample has a given spatial position relative to the target sample.

It can be understood that step S1306 may be performed by video decoder 200 in FIG. one.

For example, the target sample is a sample in the prediction block of the image block to be processed, and the prediction value of the target sample can be determined based on the sample value in the reference block.

For example, the reference sample may be a reconstructed sample that is spatially adjacent to the current CU (image block to be processed). In particular, the reference sample may be a reconstructed sample in a block other than the current CU in the picture. For example, the reference sample may be a reconstructed sample in a CU above or to the left of the current CU. This is not limited to this embodiment of the present invention.

It can be understood that, in step S1306, spatial filtering is performed on the prediction sample of the target sample using the reconstructed sample that is spatially adjacent to the current CU. In particular, a weighting calculation is performed for a target sample prediction sample in the current block and a reconstructed sample sample value that is spatially adjacent to the current CU to obtain an updated target sample prediction sample.

In an exemplary implementation, one or more reference samples may include a reconstructed sample that has the same horizontal coordinate as the target sample and has a given vertical coordinate difference with the target sample, or a reconstructed sample that has the same vertical coordinates as the target sample. the target sample and has a predefined horizontal difference from the target sample.

, и координаты верхней левой выборки блока изображения, подлежащего обработке, равны

, то опорная выборка целевой выборки может быть восстановленной выборкой в блоке выше или слева от блока изображения, подлежащего обработке. Если опорная выборка является восстановленной выборкой в блоке над подлежащим обработке блоком изображения, поскольку опорная выборка представляет собой восстановленную выборку в блоке, отличном от блока изображения, подлежащего обработке, то вертикальная координата опорной выборки представляет собой значение, полученное путем вычитания заданного отношения положения N из вертикальной координаты верхней стороны блока изображения, подлежащего обработке, и горизонтальная координата опорной выборки такая же, как горизонтальная координата целевой выборки в блоке изображения, подлежащего обработке. Точнее, координаты опорной выборки равны

. Если опорная выборка является восстановленной выборкой в блоке слева от подлежащего обработке блока изображения, поскольку опорная выборка является восстановленной выборкой в блоке, отличном от блока изображения, подлежащего обработке, горизонтальная координата опорной выборки является значением, полученным путем вычитания заданного отношения положения M из меньшей левой горизонтальной координаты блока изображения, подлежащего обработке, и вертикальная координата опорной выборки совпадает с вертикальной координатой целевой выборки в блоке изображения, подлежащего обработке. Точнее, координаты опорной выборки равны

. Конкретные отношения пространственной позиции (конкретные значения M и N) между опорной выборкой и целевой выборкой не ограничиваются в этом варианте осуществления настоящего изобретения.For example, as shown in FIG. 14, the upper left corner of the image is used as the origin of the coordinate system, the X-axis direction of the coordinate system extends to the right along the top side of the image, and the Y-axis direction of the coordinate system extends down the left side of the image. If the coordinates of the target sample in the image block to be processed (the current CU) are

, and the coordinates of the upper left sample of the image block to be processed are

, then the reference sample of the target sample may be the reconstructed sample in the block above or to the left of the image block to be processed. If the reference sample is a reconstructed sample in a block above the image block to be processed, since the reference sample is a reconstructed sample in a block other than the image block to be processed, then the vertical coordinate of the reference sample is the value obtained by subtracting the given position ratio N from the vertical the coordinates of the top side of the image block to be processed, and the horizontal coordinate of the reference sample is the same as the horizontal coordinate of the target sample in the image block to be processed. More precisely, the coordinates of the reference sample are equal to

. If the reference sample is a reconstructed sample in a block to the left of the image block to be processed, since the reference sample is a reconstructed sample in a block other than the image block to be processed, the horizontal coordinate of the reference sample is the value obtained by subtracting the specified position ratio M from the smaller left horizontal the coordinates of the image block to be processed and the vertical coordinate of the reference sample is the same as the vertical coordinate of the target sample in the image block to be processed. More precisely, the coordinates of the reference sample are equal to

. Specific spatial position relationships (specific values of M and N) between the reference sample and the target sample are not limited in this embodiment of the present invention.

In an exemplary implementation, the target sample prediction value may be updated according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляют собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, w1, w2, w3, w4, w5 и w6 являются заданными константами и M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

are the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, w1, w2, w3, w4, w5 and w6 are given constants and M1 and M2 are given positive integers.

левой верхней выборки блока изображения, подлежащего обработке.Specific methods for calculating an updated target sample prediction value based on different coordinate cases are described below.

upper left sample of the image block to be processed.

была закодирована и восстановлена, обновленное значение предсказания целевой выборки может быть получено согласно следующей формуле :In the first case, if xN is greater than 0, yN is 0, and the reference sample at position

has been encoded and reconstructed, the updated target sample prediction value can be obtained according to the following formula:

блока изображения, подлежащего обработке (CU 1) равна

, и координаты целевой выборки

в блоке изображения, подлежащего обработке, равны

. Поскольку горизонтальная координата xN левой верхней выборки текущего CU (CU 1) больше 0 и вертикальная координата yN верхней левой выборки текущей CU (CU 1) равна 0, можно определить, что текущий CU расположен в верхней части изображения. Когда выполняется пространственная фильтрация на целевой выборке в текущем CU, поскольку текущий CU расположен в верхней части изображения, и нет восстановленной выборки выше текущего CU, опорная выборка является восстановленной выборкой слева от текущего CU. Опорная выборка имеет ту же вертикальную координату, что и целевая выборка, и может обозначаться как

, где M1 представляет собой предварительно заданную пространственную позицию между опорной выборкой и целевой выборкой. Здесь, что M1 равно 1, используется в качестве примера для описания. Когда M1 равно 1, опорная выборка целевой выборки

может быть

. Таким же образом любой опорной выборкой целевой выборки

может быть

.For example, as shown in FIG. 15 that the size of the image block to be processed is 16 x 16 is used as an example for description. If the image block to be processed is CU 1, the upper left sample

image block to be processed (CU 1) is equal to

, and target sample coordinates

in the image block to be processed are

. Since the horizontal xN coordinate of the top left sample of the current CU (CU 1) is greater than 0 and the vertical coordinate yN of the top left sample of the current CU (CU 1) is 0, it can be determined that the current CU is located at the top of the image. When spatial filtering is performed on a target sample in the current CU, since the current CU is located at the top of the image and there is no reconstructed sample above the current CU, the reference sample is the reconstructed sample to the left of the current CU. The reference sample has the same vertical coordinate as the target sample and can be denoted as

, where M1 is a predefined spatial position between the reference sample and the target sample. Here, that M1 is 1 is used as an example for description. When M1 is 1, the reference sample of the target sample

may be

. In the same way, any reference sample of the target sample

may be

.

была закодирована и восстановлена, вычисление взвешивания может быть выполнено для восстановленного значения

опорной выборки и значения

предсказания целевой выборки для получения обновленного значения

предсказания целевой выборки.If the reference sample is in position

has been encoded and recovered, a weighting calculation can be performed on the recovered value

reference sample and values

target sample predictions to get updated value

target sample predictions.

была закодирована и восстановлена, обновленное значение предсказания целевой выборки может быть получено согласно следующей формуле:In the second case, if xN is 0, yN is greater than 0, and the reference sample is at position

has been encoded and reconstructed, the updated target sample prediction value can be obtained according to the following formula:

блока изображения, подлежащего обработке (CU 2), равна

, и координаты целевой выборки

равны

. Поскольку горизонтальная координата xN левой верхней выборки текущего CU (CU 2) равна 0 и вертикальная координата yN верхней левой выборки текущего CU (CU 2) больше 0, можно определить, что текущий CU расположен в левой части изображения. Когда выполняется пространственная фильтрация на целевой выборке в текущем CU, поскольку текущая CU расположена в левой части изображения, и восстановленная выборка отсутствует слева текущего CU, опорная выборка является восстановленной выборкой выше текущего CU. Опорная выборка имеет ту же горизонтальную координату, что и целевая выборка, и может быть обозначен как

, где M2 представляет собой предварительно заданную пространственную позицию между опорной выборкой и целевой выборкой. Здесь в качестве примера для описания используется то, что M2 равно 1. Когда M2 равно 1, опорной выборки целевой выборки

может быть

. Таким же образом любая опорная выборка целевой выборки

может быть

.For example, as shown in FIG. 15, if the image block to be processed is CU 2, the upper left sample

image block to be processed (CU 2) is equal to

, and target sample coordinates

equal

. Since the horizontal xN coordinate of the top left sample of the current CU (CU 2) is 0 and the vertical coordinate yN of the top left sample of the current CU (CU 2) is greater than 0, it can be determined that the current CU is located on the left side of the image. When spatial filtering is performed on the target sample in the current CU, since the current CU is located on the left side of the image and there is no reconstructed sample on the left of the current CU, the reference sample is the reconstructed sample above the current CU. The reference sample has the same horizontal coordinate as the target sample and can be denoted as

where M2 is a predefined spatial position between the reference sample and the target sample. Here, M2 is equal to 1 as an example for description. When M2 is equal to 1, the reference sample of the target sample

may be

. In the same way, any reference sample of the target sample

may be

.

была закодирована и восстановлена, вычисление взвешивания может быть выполнено для восстановленного значения

опорной выборки и значения

предсказания целевой выборки, для получения обновленного значения

предсказания целевой выборки.If the reference sample is in position

has been encoded and recovered, a weighting calculation can be performed on the recovered value

reference sample and values

target sample predictions to get an updated value

target sample predictions.

и

были закодированы и восстановлены, обновленное значение предсказания целевой выборки может быть получено согласно следующей формуле:In the third case, if xN is greater than 0, yN is greater than 0, and the reference samples at positions

and

have been encoded and reconstructed, the updated target sample prediction value can be obtained according to the following formula:

блока изображения, подлежащего обработке (CU 3), равна

, и координаты целевой выборки

равны

. Поскольку горизонтальная координата xN левой верхней выборки текущего CU (CU 3) больше 0 и вертикальная координата yN верхней левой выборки текущего CU (CU 3) больше 0, можно определить, что текущий CU не расположен на краю изображения. Когда выполняется пространственная фильтрация для целевой выборки в текущем CU, опорные выборки могут быть восстановленной выборкой над текущим CU и восстановленная выборка слева от текущего CU. Когда опорная выборка является восстановленной выборкой слева от текущего CU, восстановленная выборка имеет ту же вертикальную координату, что и целевая выборка, и может быть

; и, когда опорная выборка является восстановленной выборкой выше текущего CU, восстановленная выборка имеет ту же горизонтальную координату, что и целевая выборка, и может быть

, где M1 и M2 каждый представляют собой заданную пространственную позицию отношения между каждой из опорных выборок и целевой выборкой. Здесь то, что и M1, и M2 равны 1, используется в качестве примера для описания. Когда и M1, и M2 равны 1, опорными выборками целевой выборки

может быть

и

. Аналогично, любые опорные выборки целевой выборки

могут быть

и

.For example, as shown in FIG. 15, if the image block to be processed is CU 3, the upper left sample

image block to be processed (CU 3) is equal to

, and target sample coordinates

equal

. Since the horizontal xN coordinate of the top left sample of the current CU (CU 3) is greater than 0 and the vertical coordinate yN of the top left sample of the current CU (CU 3) is greater than 0, it can be determined that the current CU is not located at the edge of the image. When spatial filtering is performed on a target sample in the current CU, the reference samples may be the reconstructed sample above the current CU and the reconstructed sample to the left of the current CU. When the reference sample is the reconstructed sample to the left of the current CU, the reconstructed sample has the same vertical coordinate as the target sample and may be

; and, when the reference sample is a reconstructed sample above the current CU, the reconstructed sample has the same horizontal coordinate as the target sample and may be

, where M1 and M2 each represent a given spatial position of the relationship between each of the reference samples and the target sample. Here, that both M1 and M2 are equal to 1 is used as an example for description. When both M1 and M2 are 1, the reference samples of the target sample

may be

and

. Similarly, any reference samples of the target sample

may be

and

.

и

были закодированы и восстановлены, вычисление взвешивания может быть выполнено для восстановленных значений

и

опорных выборок и значения

предсказания целевой выборки, для получения обновленного значения

предсказания целевой выборки.If the reference samples at the positions

and

have been encoded and recovered, a weighting calculation can be performed on the recovered values

and

reference samples and values

target sample predictions to get an updated value

target sample predictions.

It should be noted that the values of the weighted coefficients w1, w2, w3, w4, w5 and w6, as well as the values of M1 and M2 are not limited in this embodiment of the present invention, and that both M1 and M2 are equal to 1 is used simply as an example.

For example, a set of weighted coefficients (w1, w2), (w3, w4), or (w5, w6, w7) could be a combination of w1 + w2, w3 + w4, or w5 + w6 + w7. where w1 + w2, w3 + w4 or w5 + w6 + w7 is 2 to the power of an integer. Thus, the division operation is no longer performed. For example, a combination of values such as (6, 2), (5, 3), (4, 4), (6, 1, 1) or (5, 2, 1) may be used. This is not limited in this embodiment of the present invention, and the combination of values listed herein is simply used as an example for description.

In another possible implementation, the target sample prediction value may be updated according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, w1, w2 и w3 являются заданными константами и M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, w1, w2 and w3 are given constants and M1 and M2 are given positive integers.

For example, the set of weighted coefficients (w1, w2, w3) could be a combination of the values w1 + w2 + w3, where w1 + w2 + w3 is equal to 2 raised to the power of an integer. Thus, the division operation is no longer performed. For example, a combination of values such as (6, 1, 1) or (5, 2, 1) may be used. This is not limited in this embodiment of the present invention, and the combination of values listed herein is simply used as an example for description.

It should be noted that the difference between this implementation and the previous implementation is that fetching the left of the image block to be processed and fetching above the image block to be processed are not considered in this implementation. When this implementation is used to update the target sample prediction value and the recovered reference sample values are not available, the method in the following steps S13061 and S13062 can be used to obtain a new reference sample and update the target sample prediction value based on the new standard sample.

In another possible implementation, the target sample prediction value may be updated according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координатах, соответственно, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 и w11 являются заданными константами и M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 and w11 are given constants and M1, M2, M3 and M4 are given positive integers.

верхней левой выборки блока изображения, подлежащего обработке.The following specifically describes methods for calculating an updated target sample prediction value based on different coordinate cases.

upper left sample of the image block to be processed.

и

были закодированы и восстановлены, обновленное значение предсказания целевой выборки может быть получено согласно следующей формуле:In the first case, if xN is greater than 0, yN is equal to 0, and the reference samples at positions

and

have been encoded and reconstructed, the updated target sample prediction value can be obtained according to the following formula:

блока изображения, подлежащего обработке (CU 1) равна

, и координаты целевой выборки

в блоке изображения, подлежащего обработке, равны

. Опорные выборки представляют собой восстановленные выборки слева от текущего CU. Каждая опорная выборка имеет ту же вертикальную координату, что и целевая выборка, и может быть обозначена как

и

, где M1 и M2 представляет собой заданную пространственную позицию отношения между каждой из опорных выборок и целевой выборкой. Здесь в качестве примера для описания используется то, что M1 равно 1 и M2 равно 2. Когда M1 равно 1 и M2 равно 2, опорные выборки целевой выборки

могут быть

и

. Аналогично, любые опорные выборки целевой выборки

могут быть

и

.It can be understood that the difference between this case in this implementation and the first case in the first implementation is that two reference samples are used in this implementation. If the image block to be processed is CU 1, the upper left sample

image block to be processed (CU 1) is equal to

, and target sample coordinates

in the image block to be processed are

. The pivot samples are the reconstructed samples to the left of the current CU. Each reference sample has the same vertical coordinate as the target sample and can be denoted as

and

, where M1 and M2 is the specified spatial position of the relationship between each of the reference samples and the target sample. Here, M1 is 1 and M2 is 2 as an example for description. When M1 is 1 and M2 is 2, the reference samples of the target sample

may be

and

. Similarly, any reference samples of the target sample

may be

and

.

и

были закодированы и восстановлены, вычисление взвешивания может быть выполнено для восстановленных значений

и

опорных выборок и значения

предсказания целевой выборки, для получения обновленного значения

предсказания целевой выборки.If pivot samples are in positions

and

have been encoded and recovered, a weighting calculation can be performed on the recovered values

and

reference samples and values

target sample predictions to get an updated value

target sample predictions.

и

были закодированы и восстановлены, обновленное значение предсказания целевой выборки может быть получено согласно следующей формуле:In the second case, if xN is equal to 0, yN is greater than 0, and the reference samples at positions

and

have been encoded and reconstructed, the updated target sample prediction value can be obtained according to the following formula:

блока изображения, подлежащего обработке (CU 2), равна

, и координаты целевой выборки

равны

. Опорные выборки представляют собой восстановленные выборки выше текущего CU. Каждая опорная выборка имеет такую же горизонтальную координату, что и целевая выборка, и может быть обозначена как

и

, где M3 и M4 представляют собой заданную пространственную позицию отношения между каждой из опорных выборок и целевой выборкой. Здесь в качестве примера для описания используется то, что M3 равно 1 и M4 равно 2. Когда M3 равно 1 и M4 равно 2, опорные выборки целевой выборки

могут быть

и

. Аналогично, любые опорные выборки целевой выборки

могут быть

и

.For example, the difference between this case in this implementation and the second case in the first implementation is that there are two reference samples in this implementation. If the image block to be processed is CU 2, the upper left sample

image block to be processed (CU 2) is equal to

, and target sample coordinates

equal

. The reference samples are the reconstructed samples above the current CU. Each reference sample has the same horizontal coordinate as the target sample and can be denoted as

and

, where M3 and M4 represent the specified spatial position of the relationship between each of the reference samples and the target sample. Here, M3 is 1 and M4 is 2 as an example for description. When M3 is 1 and M4 is 2, the reference samples of the target sample

may be

and

. Similarly, any reference samples of the target sample

may be

and

.

и

были закодированы и восстановлены, выполняться вычисление взвешивания для восстановленных значений

и

опорных выборок и значения

предсказания целевой выборки, для получения обновленного значения предсказания целевой выборки.If the reference samples at the positions

and

have been encoded and recovered, a weighting calculation is performed for the recovered values

and

reference samples and values

prediction of the target sample, to obtain an updated value of the prediction of the target sample.

,

,

и

были закодированы и восстановлены, обновленное значение предсказания целевой выборки может быть получено согласно следующей формуле:In the third case, if xN is greater than 0, yN is greater than 0, and the reference samples at positions

,

,

and

have been encoded and reconstructed, the updated target sample prediction value can be obtained according to the following formula:

блока изображения, подлежащего обработке (CU 3), равна

, и координаты целевой выборки

равны

. Когда опорные выборки являются восстановленными выборками слева от текущего CU, каждая восстановленная выборка имеет ту же вертикальную координату, что и целевая выборка, и может быть

и

; и, когда опорные выборки являются восстановленными выборками выше текущего CU, каждая восстановленная выборка имеет ту же горизонтальную координату, что и целевая выборка, и может быть

и

, где M1, M2, M3 и M4 каждая представляет собой предварительно заданную пространственную позицию отношений между каждой опорной выборкой и целевой выборкой. Здесь, что оба M1 и M3 равны 1, и оба M2 и M4 равны 2, используется в качестве примера для описания. Когда и M1 и M3 равны 1, и оба M2 и M4 равны 2, опорными выборками целевой выборки

могут быть

,

,

и

. Аналогично, любые опорные выборки целевой выборки

могут быть

,

,

и

.For example, the difference between this case in this implementation and the third case in the first implementation is that there are two recovered samples as pivot samples above the current CU and two recovered samples as pivot samples to the left of the current CU in this implementation. If the image block to be processed is CU 3, then the upper left sample

image block to be processed (CU 3) is equal to

, and target sample coordinates

equal

. When the reference samples are the reconstructed samples to the left of the current CU, each reconstructed sample has the same vertical coordinate as the target sample and may be

and

; and, when the reference samples are recovered samples above the current CU, each recovered sample has the same horizontal coordinate as the target sample and may be

and

, where M1, M2, M3 and M4 each represents a predetermined spatial position of the relationship between each reference sample and the target sample. Here, that both M1 and M3 are equal to 1, and both M2 and M4 are equal to 2, is used as an example for description. When both M1 and M3 are 1 and both M2 and M4 are 2, the reference samples of the target sample

may be

,

,

and

. Similarly, any reference samples of the target sample

may be

,

,

and

.

,

,

и

были закодированы и восстановлены, вычисление взвешивания может быть выполнено для восстановленных значений

,

,

и

опорных выборок и значения

предсказания целевой выборки для получения обновленного значения

предсказания целевой выборки.If the reference samples at the positions

,

,

and

have been encoded and recovered, a weighting calculation can be performed on the recovered values

,

,

and

reference samples and values

target sample predictions to get updated value

target sample predictions.

It should be noted that the values of the weighted coefficients w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 and w11 and the values of M1, M2, M3 and M4 are not limited in the embodiment of the present invention. That both M1 and M3 are equal to 1 and M2 and M4 are equal to 2 is simply used as an example for description. It can be understood that in an actual application, the values of M1 and M3 may be the same or different, the values of M2 and M4 may be the same or different, the values of M1 and M2 may be different, and the values of M3 and M4 may be different.

For example, a set of weighted coefficients (w1, w2, w3), (w4, w5, w6), or (w7, w8, w9, w10, w11) could be a combination of the values w1 + w2 + w3, w4 + w5 + w6, or w7 + w8 + w9 + w10 + w11, where w1 + w2 + w3, w4 + w5 + w6 or w7 + w8 + w9 + w10 + w11 equals 2 to the power of an integer. Thus, the division operation is no longer performed. For example, a combination of values such as (6, 1, 1), (5, 2, 1), or (3, 2, 1, 1, 1) may be used. This is not limited in this embodiment of the present invention, and the combination of values listed herein is simply used as an example for description.

In another implementation, the target sample prediction value may be updated according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляют собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

, соответственно, w1, w2, w3, w4 и w5 являются заданными константами и M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

are the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

, respectively, w1, w2, w3, w4 and w5 are given constants and M1, M2, M3 and M4 are given positive integers.

For example, a set of weighted coefficients (w1, w2, w3, w4, w5) could be a combination of the values w1 + w2 + w3 + w4 + w5, where w1 + w2 + w3 + w4 + w5 equals 2 to the power of an integer. Thus, the division operation is no longer performed. For example, a combination of values such as (3, 2, 1, 1, 1) may be used. This is not limited in this embodiment of the present invention, and the combination of values listed herein is simply used as an example for description. It should be noted that the difference between this implementation and the previous implementation is that the fetch to the left of the image block to be processed and the fetch above the image block to be processed are not considered in this implementation. When this implementation is used to update the target sample prediction value and the recovered reference sample values are not available, the method in the following steps S13061 and S13062 may be used to obtain an available reference sample and update the target sample prediction value based on the available reference sample.

In a possible implementation, one or more reference samples include one or more of the following samples: a reconstructed sample that has the same horizontal coordinate as the target sample and that is adjacent to the top side of the future image block to be processed, a reconstructed sample that has the same vertical coordinate as the target sample and which is adjacent to the left side of the image block to be processed, the top right reconstructed sample of the image block to be processed, the reconstructed sample in the lower left corner of the image block to be processed, or the reconstructed sample in upper left corner of the image block to be processed.

In another possible implementation, the target sample prediction value may be updated according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2)

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевого пикселя и w1 и w2 являются заданными константами.where the coordinates of the target pixel are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target pixel, and w1 and w2 are given constants.

выборки предсказания может быть сначала получено на основании соседней пространственной выборки и планарного (PLANAR) режима внутреннего предсказания. Можно понять, что в режиме PLANAR используются два линейных фильтра в горизонтальном направлении и вертикальном направлении, и среднее значение, полученное двумя линейными фильтрами, используется в качестве значения предсказания выборки в текущем блоке.In particular, the second meaning

prediction samples may be first obtained based on adjacent spatial sampling and planar (PLANAR) intra prediction mode. It can be understood that in the PLANAR mode, two linear filters are used in the horizontal direction and the vertical direction, and the average value obtained by the two linear filters is used as the sample prediction value in the current block.

выборки предсказания может быть получено на основании PLANAR режима:For example, the second value

prediction samples can be obtained based on the PLANAR mode:

,

,

,

и

представляют собой восстановленные значения опорных выборок на позициях

,

,

и

координат соответственно, и nTbW и nTbH представляют ширину и высота текущего CU (блока изображения, подлежащего обработке).where, as shown in FIG. 16, the coordinates of the upper left sample of the image block to be processed are

,

,

,

and

are the restored values of the reference samples at the positions

,

,

and

coordinates, respectively, and nTbW and nTbH represent the width and height of the current CU (image unit to be processed).

In another possible implementation, the target sample prediction value is updated according to the following formula:

predQ (xP, yP) = (w1 * predP (xP, yP) + w2 * predV (xP, yP) + w3 * predH (xP, yP) + ((w1 + w2 + w3) / 2)) / (w1 + w2 + w3)

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH) + nTbH / 2) >> Log2 (nTbH), predH (xP, yP) = ((nTbW - 1 - xP) * p (-1, yP) + (xP + 1) * p (nTbW, -1) + nTbW / 2) >> Log2 (nTbW), coordinates of the target sample (xP, yP), the coordinates of the upper left sample of the image block to be processed are (0, 0), predP (xP, yP) represents the prediction value of the target sample before updating, predQ (xP, yP) represents the updated prediction value of the target samples, p(xP, –1), p(–1, nTbH), p(–1, yP), and p(nTbW, –1) represent the reconstructed values of the reference samples in coordinates (xP, –1), (–1 , nTbH), (-1, yP) and (nTbW, -1), respectively, w1 and w2 are given constants, and nTbW and nTbH represent the width and height of the image block to be processed.

In another possible implementation, the target sample prediction value is updated according to the following formula:

predQ (xP, yP) = (((w1 * predP (xP, yP)) << (Log2 (nTbW) + Log2 (nTbH) + 1)) + w2 * predV (xP, yP) + w3 * predH (xP , yP) + (((w1 + w2 + w3) / 2) << (Log2 (nTbW) + Log2 (nTbH) + 1))) / (((w1 + w2 + w3) << (Log2 (nTbW) + Log2 (nTbH) + 1)))

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH)) << Log2 (nTbW), predH (xP, yP) = ((nTbW - 1 - xP) * p (–1, yP) + (xP + 1) * p (nTbW, –1)) << Log2 (nTbH), target sample coordinates (xP, yP), the top left sample coordinates of the image block to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, -1 ), p (–1, nTbH), p (–1, yP), and p (nTbW, –1) represent the reconstructed values of the reference samples at positions (xP, –1), (–1, nTbH), (–1, yP) and (nTbW, –1) coordinates, respectively, w1 and w2 are given constants, and nTbW and nTbH represent the width and height of the image block to be processed.

выборки предсказания, не ограничивается алгоритмом в VTM, и альтернативно может использоваться алгоритм PLANAR в HEVC или H.264. Это не ограничивается данным вариантом выполнения настоящего изобретения.It should be noted that the PLANAR mode algorithm used to generate the second value

prediction sampling is not limited to the algorithm in VTM, and the PLANAR algorithm in HEVC or H.264 may alternatively be used. This is not limited to this embodiment of the present invention.

It should be noted that the values of the weighted coefficients w1 and w2 are not limited in this embodiment of the present invention. For example, the set of weighted coefficients (w1, w2) could be a combination of the values w1 + w2, where w1 + w2 is equal to 2 raised to the power of an integer. Thus, the division operation is no longer performed. For example, a combination of values such as (6, 2), (5, 3) or (4, 4) may be used. This is not limited in this embodiment of the present invention, and the combination of values listed herein is simply used as an example for description.

In another possible implementation, the target sample prediction value may be updated according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2)

where

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

координат, соответственно, w1 и w2 являются заданными константами и nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

coordinates, respectively, w1 and w2 are given constants and nTbW and nTbH represent the width and height of the image block to be processed.

предсказания целевой выборки с использованием способа DC режима в технологии обработки комбинации с внутренним предсказанием в VTM. Можно понять, что при использовании DC режима значение предсказания текущего блока может быть получено на основании среднего значения опорных выборок слева и над текущим блоком.In another exemplary implementation, an inter prediction block may be processed using a position-dependent intra prediction combination process that is used in intra prediction, and an updated value may be obtained.

predicting the target sample using the DC mode method in the VTM intra-prediction combination processing technology. It can be understood that when using the DC mode, the prediction value of the current block can be obtained based on the average of the reference samples left and above the current block.

предсказания целевой выборки может быть получено согласно следующей формуле:For example, the updated value

target sample prediction can be obtained according to the following formula:

where

,

,

, как показано на фиг. 16, координаты целевой выборки равны

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

и

представляют восстановленные значения опорных выборок на позициях

,

и

, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, clip1Cmp представляет операцию отсечения.

,

,

, as shown in FIG. 16, the coordinates of the target sample are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

and

represent the reconstructed values of the reference samples at the positions

,

and

, respectively, nTbW and nTbH represent the width and height of the image block to be processed, clip1Cmp represents the clipping operation.

предсказания целевой выборки может быть получено с использованием не только технологии обработки комбинации внутреннего предсказания в VTM, но также алгоритма в JEM.Note that the updated value

The target sample prediction can be obtained using not only the intra prediction combination processing technology in VTM, but also the algorithm in JEM.

предсказания целевой выборки может быть получено с использованием режима PLANAR в технологии обработки комбинации внутреннего предсказания в VTM. Используется следующая формула:In an exemplary implementation, the inter prediction block may be processed using the position-dependent intra prediction combination processing technique that is used in intra prediction, and the updated value

The target sample prediction can be obtained using the PLANAR mode in the VTM intra prediction combination processing technology. The following formula is used:

where,

,

, как показано на фиг. 16, координаты целевой выборки равны

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки до обновления,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, clip1Cmp представляет операцию отсечения.

,

, as shown in FIG. 16, the coordinates of the target sample are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

, respectively, nTbW and nTbH represent the width and height of the image block to be processed, clip1Cmp represents the clipping operation.

предсказания целевой выборки может быть получено с использованием не только алгоритма в VTM, но также алгоритма в JEM.Note that the updated value

target sample prediction can be obtained using not only the algorithm in VTM, but also the algorithm in JEM.

In an exemplary implementation, filtering processing may be performed on the inter prediction sample using intra prediction edge filtering technology. See the HEVC method for details on edge filtering technology. Details are not described here.

It should be noted that when the target sample prediction value is updated in any of the above methods, if the recovered reference sample value is not available, step S1306 may further include the following steps S13061 and S13062.

S13061: When the recovered reference sample value is not available, determine in a predetermined order the availability of samples adjacent to the top and left side of the image block to be processed until the predetermined number of available reference samples is obtained.

; или, когда блок изображения, подлежащий обработке, расположен на левой стороне изображения, восстановленное значение опорной выборки отсутствует на координатной позиции

; или значение реконфигурации не может быть получено для опорной выборки. Конкретный случай, в котором восстановленное значение опорной выборки недоступно, не ограничивается в этом варианте осуществления настоящего изобретения, и случай в данном документе просто используется в качестве примера для описания.It can be understood that the case in which the restored reference sample value is not available may include the following: when the image block to be processed is located on the top side of the image, the restored reference sample value is not available at the coordinate position

; or, when the image block to be processed is located on the left side of the image, the reconstructed reference sample value is not present at the coordinate position

; or the reconfiguration value cannot be obtained for the reference sample. The specific case in which the recovered reference sample value is not available is not limited in this embodiment of the present invention, and the case herein is simply used as an example for description.

In the implementation shown in FIG. 17, the given order can be the order from the coordinates (xN - 1, yN + nTbH - 1) to the coordinates (xN - 1, yN - 1), and then from the coordinates (xN, yN - 1) to the coordinates (xN + nTbW - 1, yN - 1). For example, all samples can be traversed in order from coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinate (xN + nTbW - 1, yN - 1) to search for an available reference sample from the samples adjacent to the top and left side of the image block to be processed. It should be noted that the specific predetermined order is not limited in this embodiment of the present invention, and the predetermined order is used here simply as an example for description.

For example, when there is at least one pivot sample available in all pivot samples, if the reconstructed pivot sample value (xN - 1, yN + nTbH - 1) is not available, the available sample is searched in the given order from coordinates (xN - 1, yN + nTbH - 1) to coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1). As soon as an available sample is found, the search ends. If the available sample is (x, y), the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is set equal to the reconstructed sample value (x, y). If the reconstructed reference sample value (x, y) is not available in the set including the reference sample (xN - 1, yN + nTbH - M), the reconstructed reference sample value (x, y) is set to the reconstructed sample value (x, y + 1), where M is greater than or equal to 2 and less than or equal to nTbH + 1. If the reconstructed reference sample value (x, y) is not available in the set including the reference sample (xN + N, yN - 1), the reconstructed reference sample value (x, y) is set to the reconstructed value of the reference sample (x - 1, y), where N is greater than or equal to 0 and less than or equal to nTbW - 1.

For example, if the reconstructed reference sample value (xN - 1, yN + nTbH - M) is not available, the available reference sample can be found in the given order, starting at coordinates (xN - 1, yN + nTbH - M), where M is greater than or equal to 1 and less than or equal to nTbH + 1. If the available reference sample is B, the reconstructed reference sample value (xN - 1, yN + nTbH - M) can be set to the reconstructed reference sample value B. If the reconstructed reference sample value with coordinates (xN + N, yN - 1) is not available, the available reference sample can be found in the specified order, starting at the coordinate (xN + N, yN - 1) where N is greater than or equal to 0 and less than or equal to nTbW - 1. If the available reference sample is C, the reconstructed reference sample value (xN + N, yN - 1) can be set to the reconstructed reference sample value C.

For example, if the reconstructed reference sample value (xN - 1, yN + nTbH - 3) is not available, the availability of samples adjacent to the top and left sides of the image block to be processed can be determined in order from coordinates (xN - 1, yN + nTbH - 3) to coordinates (xN - 1, yN - 1) until the specified number of available reference samples is obtained and the restored reference sample value (xN - 1, yN + nTbH - 3) can be set to the restored value of the available reference sample. If the reconstructed reference sample value with coordinates (xN + 3, yN - 1) is not available, an available sample is searched in order from coordinates (xN + 3, yN - 1) to coordinates (xN + nTbW - 1, yN - 1), and the reconstructed the reference sample value (xN + 3, yN - 1) can be set to the recovered value of the available reference sample.

For example, if the restored value of the reference sample (xN - 1, yN + nTbH - 1) is not available, the available sample is searched in the given order from the coordinates (xN - 1, yN + nTbH - 1) to the coordinates (xN - 1, yN - 1) and then from coordinates (xN, yN - 1) to coordinates (xN + nTbW - 1, yN - 1). Once an available sample is found, the search ends. If the available sample is equal to (x, y), the reconstructed reference sample value (xN - 1, yN + nTbH - 1) is set equal to the reconstructed sample value (x, y). If the reconstructed reference sample value (xN - 1, yN + nTbH - M) is not available, you can search for the available reference sample in the reverse order of the specified one, starting from coordinates (xN - 1, yN + nTbH - M), where M is greater than 1 and less than or equal to nTbH + 1. If the available reference sample is C, the reconstructed reference sample value (xN - 1, yN + nTbH - M) can be set to the reconstructed reference sample value C. If the reconstructed reference sample value with coordinates (xN + N, yN - 1) is not available, the available reference sample can be found in reverse order, starting at (xN + N, yN - 1) where N is greater than or equal to 0 and less than or equal to nTbW - 1. If the available reference sample is D, the reconstructed reference sample value (xN + N, yN - 1) can be set equal to the reconstructed reference sample value D.

For example, if the reconstructed reference sample value (xN - 1, yN + nTbH - 3) is not available, the availability of samples adjacent to the top and left sides of the image block to be processed can be determined in order from coordinates (xN - 1, yN + nTbH - 3) to coordinates (xN - 1, yN + nTbH - 1) until the given number of available reference samples is obtained, and the restored value of the reference sample (xN - 1, yN + nTbH - 3) can be set to the restored value of the available reference sample. If the reconstructed reference sample value with coordinates (xN + 3, yN - 1) is not available, the available sample is searched in order from coordinates (xN + 3, yN - 1) to coordinates (xN, yN - 1) and the reconstructed reference sample value ( xN + 3, yN - 1) can be set to the reconstructed value of the available reference sample.

It should be noted that the new reference sample may be the first available reference sample found in the given order, or may be any available reference sample found in the given order. This is not limited to this embodiment of the present invention.

It can be understood that, according to the method, when a reference sample is unavailable, an available reference sample can be found from the samples adjacent to the top and left side of the image block to be processed in a given order, and the recovered value of the available reference sample is used as the recovered value of the unavailable reference samples.

S13062: Perform a weighting calculation on the recovered reference sample available value and the target sample prediction value to update the target sample prediction value.

For example, the target sample prediction value may be updated based on the recovered new reference sample value using any of the above implementations.

It should be noted that if the restored reference sample value is not available, and it is determined in step S13061 that all samples adjacent to the top and left side of the image block to be processed are not available, the restored reference sample value may be set to 1 << ( bitDepth - 1), where bitDepth represents the bit depth of the sample value of the reference sample. For example, when the image block to be processed is located in the upper left corner of the image, and the coordinates in the upper left corner of the image block to be processed are (0, 0), all samples adjacent to the top and left side of the image block to be processed are not available. In this case, the reconstructed value of the reference sample corresponding to the target sample in the image block to be processed may be set to 1<<(bitDepth - 1).

In the aforementioned plurality of implementations, spatial filtering is performed on the inter prediction sample during inter prediction sample generation. Compared with conventional technology, the coding efficiency is improved.

In an exemplary implementation prior to step S1306, the method may further include step S1305.

S1305: (Optional) Perform filtering processing on the reference sample.

It can be understood that step S1305 may be performed by the filter section 206 in the video decoder in FIG. 3.

For example, performing filtering processing on the reference sample may include: when the reference sample is located over an image block to be processed, performing a weighting calculation on the restored value of the reference sample and the restored values of the left neighbor sample and the right neighbor sample of the reference sample; when the reference sample is located to the left of the image block to be processed, performing a weighting calculation of the restored value of the reference sample and the restored values of the top neighbor sample and the bottom neighbor sample of the reference sample; and updating the recovered reference sample value using the weighting result.

It can be understood that after the filtering processing is performed on the reference sample in step S1305, when step S1306 is performed, the weighting calculation can be performed on the updated recovered value of the reference sample after the filtering processing and the prediction value of the target sample to update the prediction value of the target sample.

It should be noted that, for a particular method of performing the filtering processing on the reference sample, reference may be made to the filtering method in step S1306. Details are not described here.

It can be understood that filtering processing is performed on the reference sample to update the restored value of the reference sample, and filtering processing is performed on the target sample based on the updated restored value of the reference sample. This can further improve the coding efficiency and reduce the prediction residual.

In a possible implementation, before step S1306 or after step S1306, step S1307 may be additionally performed.

S1307: (Could) Continue performing inter prediction based on motion information and bitstream information using an inter coding technique other than this method.

It can be understood that step S1307 may be performed by external predictor 210 in the video decoder in FIG. 3.

For example, HEVC or VTM technologies may be used, including, but not limited to, bi-directional optical flow method, decoder side motion vector refinement method, local illumination compensation (LIC) technology, general weighted prediction (GBI), overlapping block motion compensation (OBMC). ) and Decoder Side Motion Vector Compensation (DMVD) technology. The method in HEVC or VTM, or another method, may be used to list motion vector prediction candidates. This is not limited to this embodiment of the present invention.

It should be noted that the execution order of steps S1301 to S1307 in the above method is not limited in this embodiment of the present invention. For example, step S1305 may be performed before step S1307 or may be performed after step S1307. This is not limited to this embodiment of the present invention.

In an exemplary implementation, before performing motion compensation for an image block to be processed based on the motion information, the method may further include: initially updating the motion information using a first predetermined algorithm; and accordingly, performing motion compensation of the image block to be processed based on the motion information includes: performing motion compensation of the image block to be processed based on the initially updated motion information.

In another possible implementation, after a prediction block of an image block to be processed is obtained, the method may further include: pre-updating the prediction block using a second predetermined algorithm; and accordingly, performing a weight calculation on one or more recovered values of one or more reference samples and a target sample prediction value includes: performing a weight calculation on one or more recovered values of one or more reference samples and a preliminary updated target sample prediction value.

In another possible implementation, after performing a weighting calculation on one or more recovered values of one or more reference samples and a target sample prediction value, to update the target sample prediction value, the method further includes: updating the target sample prediction value using a second predetermined algorithm.

It should also be understood that after obtaining an updated prediction value of the target sample, the method may further include: adding a final inter-prediction image and a residual image to obtain a reconstructed image of the current block. In particular, if the current block has a residual, the residual information is added to the prediction image to obtain a reconstructed image of the current block. If the current block has no remainder, the predicted image is the reconstructed image of the current block. The above process may use the same method as HEVC or VTM, or another method for motion compensation or image restoration. It's not limited.

According to the inter prediction method provided in this embodiment of the present invention, a bit stream is analyzed to obtain motion information of an image block to be processed; motion compensation is performed for the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed; and performing a weighting calculation on the one or more recovered values of the one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a given spatial position relationship with the target sample. In this embodiment of the present invention, after obtaining the prediction value of the target sample in the block of the image to be processed, filtering processing is performed on the prediction value of the target sample using an adjacent reconstructed sample. Thus, the coding compression efficiency can be improved and the PSNR BDrate is reduced by 0.5%. Compared with the conventional technology, spatial filtering performed on the inter prediction sample during inter prediction sample generation improves coding efficiency.

An embodiment of the present invention provides an inter prediction device. The device may be a video decoder. Specifically, the inter prediction device is configured to perform the steps performed by the decoding device in the above inter prediction method. The inter prediction device provided in this embodiment of the present invention may include modules corresponding to respective steps.

In this embodiment of the present invention, the inter prediction device may be divided into functional modules based on the above method examples. For example, functional modules may be obtained by separation based on respective functions, or two or more functions may be integrated into one processing module. An integrated module may be implemented in hardware or may be implemented in a software function module. In this embodiment of the present invention, the division into modules is an example, is simply a division of logical functions, and may be another division in the actual implementation.

When functional modules are obtained by separation based on respective functions, FIG. 18 shows an exemplary design of the inter predictor 1800 in the above embodiment. As shown in FIG. 18, inter predictor 1800 may include a parsing module 1801, a compensation module 1802, and a calculation module 1803. In particular, the functions of the modules are as follows:

The parsing unit 1801 is configured to perform parsing of the bitstream to obtain motion information of the image block to be processed.

The compensation unit 1802 is configured to perform motion compensation on the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed, where the prediction block of the image block to be processed contains the prediction value of the target sample.

Calculation module 1803 is configured to perform weighting calculations on one or more recovered values of one or more reference samples and a prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a predetermined spatial position relative to the target sample.

In a possible implementation, one or more reference samples include a reconstructed sample that has the same horizontal coordinate as the target sample and has a specified difference in vertical coordinates with the target sample, or a reconstructed sample that has the same vertical coordinate as the target sample. sample and has a given difference in horizontal coordinates with the target sample.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, w1, w2, w3, w4, w5 и w6 являются заданными константами, M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, w1, w2, w3, w4, w5 and w6 are given constants, M1 and M2 are given positive integers.

In a possible implementation, w1 + w2 = R, w3 + w4 = R, or w5 + w6 + w7 = R, where R is 2 to the power of n and n is a non-negative integer.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок в координатах

,

,

и

, соответственно, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 и w11 являются заданными константами, M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the restored values of the reference samples in coordinates

,

,

and

, respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10 and w11 are given constants, M1, M2, M3 and M4 are given positive integers.

In a possible implementation, w1 + w2 + w3 = S, w4 + w5 + w6 = S, or w7 + w8 + w9 + w10 + w11 = S, where S is 2 to the power of n, n is a non-negative integer.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки до обновления,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок на позициях

и

координат, соответственно, w1, w2 и w3 являются заданными константами, M1 и M2 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

and

represent the reconstructed values of the reference samples at the positions

and

coordinates, respectively, w1, w2 and w3 are given constants, M1 and M2 are given positive integers.

In a possible implementation, w1 + w2 + w3 = R, where R is 2 to the power of n, and n is a non-negative integer.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет собой значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

, соответственно, w1, w2, w3, w4 и w5 являются заданными константами, M1, M2, M3 и M4 являются заданными положительными целыми числами.where the target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

is the prediction value of the target sample before updating,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

, respectively, w1, w2, w3, w4 and w5 are given constants, M1, M2, M3 and M4 are given positive integers.

In a possible implementation, w1 + w2 + w3 + w4 + w5 = S, where S is 2 to the power of n and n is a non-negative integer.

In a possible implementation, one or more reference samples include one or more of the following samples: a reconstructed sample that has the same horizontal coordinate as the target sample and that is adjacent to the top side of the image block to be processed, a reconstructed sample that has the same vertical coordinate as the target sample and which is adjacent to the left side of the image block to be processed, the top right reconstructed sample of the image block to be processed, the reconstructed sample in the lower left corner of the image block to be processed, or the reconstructed sample in the left upper corner of the image block to be processed.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2)

where predP1 (xP, yP) = (predV (xP, yP) + predH (xP, yP) + nTbW * nTbH) >> (Log2 (nTbW) + Log2 (nTbH) + 1), predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH)) << Log2 (nTbW), predH (xP, yP) = ((nTbW - 1 - xP) * p (–1, yP) + (xP + 1) * p (nTbW, –1)) << Log2 (nTbH), target sample coordinates are (xP, yP), upper left sample coordinates of the image block, to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, –1), p(–1, nTbH), p (–1, yP) and p (nTbW, –1) represent the reconstructed values of the reference samples in the coordinates (xP, –1), (–1, nTbH), (–1, yP) and (nTbW, – 1), respectively, w1 and w2 are given constants, nTbW and nTbH represent the width and height of the image block to be processed.

In a possible implementation, the target sample prediction value is updated according to the following formula:

predQ (xP, yP) = (w1 * predP (xP, yP) + w2 * predV (xP, yP) + w3 * predH (xP, yP) + ((w1 + w2 + w3) / 2)) / (w1 + w2 + w3)

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH) + nTbH / 2) >> Log2 (nTbH), predH (xP, yP) = ((nTbW - 1 - xP) * p (-1, yP) + (xP + 1) * p (nTbW, -1) + nTbW / 2) >> Log2 (nTbW), coordinates of the target sample are (xP, yP), the coordinates of the upper left sample of the image block to be processed are (0, 0), predP (xP, yP) represents the prediction value of the target sample before updating, predQ (xP, yP) represents the updated prediction value target sample, p(xP, –1), p(–1, nTbH), p(–1, yP), and p(nTbW, –1) represent the reconstructed values of the reference samples in coordinates (xP, –1), (– 1, nTbH), (–1, yP) and (nTbW, –1), respectively, w1 and w2 are given constants, nTbW and nTbH represent the width and height of the image block to be processed.

In a possible implementation, the target sample prediction value is updated according to the following formula:

predQ (xP, yP) = (((w1 * predP (xP, yP)) << (Log2 (nTbW) + Log2 (nTbH) + 1)) + w2 * predV (xP, yP) + w3 * predH (xP , yP) + (((w1 + w2 + w3) / 2) << (Log2 (nTbW) + Log2 (nTbH) + 1))) / (((w1 + w2 + w3) << (Log2 (nTbW) + Log2 (nTbH) + 1)))

where predV (xP, yP) = ((nTbH - 1 - yP) * p (xP, –1) + (yP + 1) * p (–1, nTbH)) << Log2 (nTbW), predH (xP, yP) = ((nTbW - 1 - xP) * p (–1, yP) + (xP + 1) * p (nTbW, –1)) << Log2 (nTbH), target sample coordinates are (xP, yP) , the coordinates of the upper left sample of the image block to be processed are (0, 0), predP(xP, yP) represents the prediction value of the target sample before updating, predQ(xP, yP) represents the updated prediction value of the target sample, p(xP, - 1), p (–1, nTbH), p (–1, yP), and p (nTbW, –1) represent the reconstructed values of the reference samples at positions (xP, –1), (–1, nTbH), (–1 , yP) and (nTbW, –1) coordinates, respectively, w1 and w2 are given constants, nTbW and nTbH represent the width and height of the image block to be processed.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2)

where

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

,

и

представляют восстановленные значения опорных выборок на позициях

,

,

и

, соответственно, w1 и w2 являются заданными константами, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке.target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

, respectively, w1 and w2 are given constants, nTbW and nTbH represent the width and height of the image block to be processed.

In a possible implementation, the sum of w1 and w2 is 2 to the power of n, and n is a non-negative integer.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

where

,

,

, координаты целевой выборки равны

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

,

и

представляют восстановленные значения опорных выборок на позициях

,

и

, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, clip1Cmp представляет операцию отсечения.

,

,

, the coordinates of the target sample are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

and

represent the reconstructed values of the reference samples at the positions

,

and

, respectively, nTbW and nTbH represent the width and height of the image block to be processed, clip1Cmp represents the clipping operation.

In an exemplary implementation, calculation module 1803 updates the target sample prediction value according to the following formula:

,

координаты целевой выборки равны

, координаты верхней левой выборки блока изображения, подлежащего обработке, равны

,

представляет значение предсказания целевой выборки перед обновлением,

представляет обновленное значение предсказания целевой выборки,

и

представляют восстановленные значения опорных выборок в координатах

и

, соответственно, nTbW и nTbH представляют ширину и высоту блока изображения, подлежащего обработке, clip1Cmp представляет операцию отсечения.Where

,

target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

and

represent the restored values of the reference samples in coordinates

and

, respectively, nTbW and nTbH represent the width and height of the image block to be processed, clip1Cmp represents the clipping operation.

In an exemplary implementation, the calculation module 1803 is further configured to: when a reconstructed reference sample value is not available, determine in a predetermined order the availability of samples adjacent to the top side and left side of the image block to be processed until a predetermined number of available reference samples are obtained. samples; and perform a weighting calculation of the reconstructed value of the available reference sample and the predicted value of the target sample.

In an exemplary implementation, the calculation module 1803 is further configured to: when the reference sample is positioned over the image block to be processed, perform a weighting calculation of the restored value of the reference sample and the reconstructed values of the left neighbor sample and the right neighbor sample of the reference sample; when the reference sample is located to the left of the image block to be processed, perform a weighting calculation for the restored value of the reference sample and the restored values of the top neighbor sample and the bottom neighbor sample of the reference sample; and update the recovered reference sample value using the result of the weighting calculation.

In an exemplary implementation, the calculation module 1803 is further configured to initially update the motion information using a first predetermined algorithm. Accordingly, the compensation module 1802 is specifically configured to perform motion compensation on the image block to be processed based on the originally updated motion information.

In an exemplary implementation, calculation module 1803 is further configured to pre-update the prediction block using a second predetermined algorithm. Accordingly, calculation module 1803 is specifically configured to perform weighting calculation of one or more recovered values of one or more reference samples and the previously updated prediction value of the target sample.

In an exemplary implementation, calculation module 1803 is further configured to update the target sample prediction value using a second predetermined algorithm.

In an exemplary implementation, the parsing module 1801 is further configured to: parse the bitstream to obtain a prediction mode corresponding to the image block to be processed; and determine that the prediction mode is a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode.

In an exemplary implementation, the parsing module 1801 is further configured to: perform parsing of the bitstream to obtain update definition information of the image block to be processed; and determining that the update determination indication information is used to indicate an update of the prediction block of the image block to be processed.

In an exemplary implementation, the calculation module 1803 is further configured to: obtain predetermined update determination information of an image block to be processed; and determining that the update determination indication information is used to indicate an update of the prediction block of the image block to be processed.

Fig. 19 is a diagram of an inter prediction device 1900 according to an embodiment of the present invention. In particular, the device includes a processor 1901 and a memory 1902 that is coupled to the processor. Processor 1901 is configured to execute the embodiment and possible implementations shown in FIG. 13.

Processor 1901 may be a Central Processing Unit (CPU), general purpose processor, Digital Signal Processor (DSP), ASIC, FPGA, or other programmable logic device, transistorized logic device, hardware component, or any combination thereof. The processor 1901 may implement or execute various exemplary logical blocks, modules, and circuits described with reference to the content disclosed in the present invention. Alternatively, the processor may be a combination of processors that implement a computing function, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

All related content of each script in the above method embodiment can be cited in the function descriptions of the respective functional modules. The details are again not described here.

While specific aspects of the present invention have been described with reference to video encoder 100 and video decoder 200, it should be understood that the techniques of the present invention may be used by many other video encoding and/or decoding modules, processors, processing units, and hardware encoding units, and the like, for example , encoders/decoders (CODEC). Additionally, it should be understood that the steps shown and described in FIG. 13 are presented simply as possible implementations. In other words, the steps shown in the possible implementations in FIG. 13 are not necessarily performed in the order shown in FIG. 13 and fewer additional or alternative steps may be performed.

Furthermore, it should be understood that, depending on possible implementations, specific actions or events in any of the ways described in this specification may be performed in different sequences, an action or event may be added, or actions or events may be combined or omitted (e.g. , not all described actions or events are necessary to implement the method). In addition, in a particular possible implementation, actions or events may (for example) be threaded or interrupted, or may be processed by multiple processors simultaneously rather than sequentially. Furthermore, while specific aspects of the present invention are described as being performed by a single module or device for clarity, it should be understood that the technologies of the present invention may be performed by a combination of modules or modules associated with a video decoder.

In one or more possible implementations, the described functions may be implemented using hardware, software, firmware, or any combination thereof. If the functions are implemented using software, the functions may be stored on a computer-readable medium as one or more instructions or code, or transmitted over a computer-readable medium and executed by a hardware processing unit. The computer-readable medium may include a computer-readable storage medium or communication medium. A computer-readable storage medium corresponds to a tangible medium, such as a storage medium. A communication medium includes any medium that facilitates the transfer of a computer program (for example) from one place to another in accordance with a communication protocol.

Thus, a computer-readable medium may correspond to, for example, (1) a tangible computer-readable storage medium or (2) a communication medium such as a signal or a medium. A storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the technologies described in the present invention. The computer program product may include a computer readable medium.

As a possible implementation, and not limitation, a computer-readable storage medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage device, magnetic disk storage device or other magnetic storage device, flash memory, or any other a medium that can be used to store the required code in the form of an instruction or data structure and that can be accessed by a computer. Similarly, any connection can be appropriately referred to as a computer-readable medium. For example, if an instruction is transmitted from a website, server, or other remote source via coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, coaxial cable, fiber optic cable , twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are used as the medium.

However, it should be understood that a computer-readable storage medium and a storage medium may not include a connection, a medium, a signal, or other temporary medium, but alternatively means a permanent tangible storage medium. The magnetic disc and optical disc used in this specification include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disc and Blu-ray disc. A magnetic disk usually reproduces data magnetically, an optical disk reproduces data optically using a laser. The combination of the aforementioned magnetic disk and optical disk should also be considered as a computer-readable medium.

An instruction may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalents. integrated or discrete logic circuits. Therefore, the term "processor" as used in this specification may be any of the above structures, or another structure that is used to implement the technologies described in this specification. Additionally, in some aspects, the functions described in this specification may be provided within dedicated hardware and/or software modules capable of encoding and decoding, or may be included in a combined codec. In addition, all technologies can be implemented in one or more circuits or logic elements.

The technologies in the present invention may be implemented in various devices or devices including a wireless mobile phone, an integrated circuit (IC), or an IC stack (eg, a chipset). The present invention describes various components, modules, or blocks to emphasize the functional aspects of a device capable of performing the disclosed technologies, but need not be implemented in different hardware blocks. More specifically, as described above, the various blocks may be combined into a codec hardware block or provided with cooperating hardware blocks (including one or more of the processors described above) in combination with an appropriate set of software and/or firmware.

The above descriptions are merely specific implementations of the present invention, but are not intended to limit the protection scope of the present invention. Any change or substitution readily detectable by one of ordinary skill in the art within the technical scope disclosed in the present invention should be within the protection scope of the present invention. Therefore, the scope of protection of the present invention should be consistent with the scope of protection of the claims.

Claims (74)

1. An external prediction method, comprising: parsing the bitstream to obtain motion information of the image block to be processed; performing motion compensation for the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed, in which the prediction block of the image block to be processed contains a prediction value of the target sample; and performing a weighting calculation on one or more recovered values of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, in which the reference sample has a given spatial position relationship with the target sample, wherein, before performing the weighting calculation on the one or more recovered values of the one or more reference samples and the target sample prediction value, the method further comprises: parsing the bitstream to obtain a prediction mode corresponding to the image block to be processed; and determining that the prediction mode is a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode. 2. The method of claim 1, wherein the one or more reference samples comprise: a reconstructed sample that has the same horizontal coordinate as the target sample and has a given vertical coordinate difference with the target sample, or a reconstructed sample that has the same vertical coordinate as the target sample and has a given horizontal coordinate difference with the target sample , or a reconstructed sample that has the same horizontal coordinate as the target sample and that is near the top side of the image block to be processed, a reconstructed sample that has the same vertical coordinate as the target sample and that is adjacent to the left side of the image block to be processed, the top right reconstructed sample of the image block to be processed, the lower left reconstructed sample of the image block to be processed, or the top left reconstructed sample of the image block to be processed. 3. The method of claim 1 or 2, wherein updating the prediction value of the target sample comprises performing a weighting calculation based on the prediction value of the target sample before the update and the restored reference sample value to obtain an updated prediction value of the target sample, wherein the updated prediction value of the target sample is obtained according to the following formula:

, where

,

target sample coordinates are

, the coordinates of the upper left sample of the image block to be processed are

,

represents the prediction value of the target sample before the update,

represents the updated prediction value of the target sample,

,

,

and

represent the reconstructed values of the reference samples at the positions

,

,

and

, respectively, w1 and w2 are given constants and nTbW and nTbH represent the width and height of the image block to be processed. 4. The method of claim 3, wherein the sum of w1 and w2 is 2 to the power of n and n is a non-negative integer. 5. The method according to any one of paragraphs. 1-4, wherein performing a weighting calculation on one or more recovered values of one or more reference samples and a target sample prediction value comprises: when the recovered reference sample value is not available, determining in a given order the availability of samples adjacent to the top and left side of the image block to be processed until a given number of available reference samples is obtained; and performing a weighting calculation of the recovered value of the available reference sample and the predicted value of the target sample. 6. The method according to any one of paragraphs. 1-5, wherein, before performing motion compensation of the image block to be processed based on the motion information, the method further comprises: initially updating the motion information using the first predetermined algorithm; and accordingly, performing motion compensation for the image block to be processed based on the motion information comprises: performing motion compensation for the image block to be processed based on the initially updated motion information. 7. The method according to any one of paragraphs. 1-6, wherein after obtaining a prediction block of an image block to be processed, the method further comprises: pre-updating the prediction block using a second predetermined algorithm; and accordingly, performing a weighting calculation on one or more recovered values of one or more reference samples and a predicted target sample value comprises: performing a weighting calculation on the one or more recovered values of the one or more reference samples and the previously updated prediction value of the target sample. 8. The method according to any one of paragraphs. 1-6, wherein after performing the weighting calculation on one or more recovered values of one or more reference samples and the target sample prediction value to update the target sample prediction value, the method further comprises: updating the prediction value of the target sample using the second predetermined algorithm. 9. The method according to any one of paragraphs. 1-8, wherein, before performing the weighting calculation on one or more recovered values of one or more reference samples and the target sample prediction value, the method further comprises: parsing the bit stream to obtain information indicating the update definition of the image block to be processed; and determining that the update determination indication information is used to indicate to update the prediction block of the image block to be processed. 10. A prediction method, comprising: obtaining a first prediction value of the target sample by inter prediction; obtaining a second prediction value of the target sample by intra prediction; and performing a weighting calculation on the first prediction value and the second prediction value to obtain an updated prediction value of the target sample, in which the weighted coefficient of the first prediction value differs from the weighted coefficient of the second prediction value, wherein, before performing the weighting calculation for the first prediction value and the second prediction value, the method further comprises: parsing the bitstream to obtain a prediction mode corresponding to the image block to be processed; and determining that the prediction mode is a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode, wherein the prediction block of the image block to be processed contains the first prediction value of the target sample. 11. The method of claim 10, wherein the second prediction value is obtained based on a spatial neighbor sample using a planar (PLANAR) intra prediction mode. 12. The method of claim 10, wherein obtaining the first prediction value of the target sample by inter-prediction comprises: performing bitstream parsing to obtain motion information of the image block to be processed; and performing motion compensation for the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed. 13. The method according to any one of paragraphs. 10-12, wherein the updated prediction value predQ(xP, yP) is obtained based on w1*predP(xP, yP) and w2*predP1(xP, yP), where (xP, yP) represents target sample coordinates, predP (xP, yP) represents the first prediction value, predP1 (xP, yP) represents the second prediction value, w1 represents the weighted coefficient of the first prediction value, w2 represents the weighted coefficient of the second prediction value, w1 and w2 are given constants and w1 is not equal to w2. 14. The method of claim 13, wherein the set (w1, w2) of weighted coefficients is (6, 2) or (5, 3). 15. The method of claim 13, wherein the sum of w1 and w2 is 2 raised to the power of n, and n is a non-negative integer. 16. An external prediction device, comprising: a parsing module, configured to parse the bitstream to obtain motion information of the image block to be processed; a compensation module, configured to perform motion compensation for the image block to be processed based on the motion information to obtain a prediction block of the image block to be processed, in which the prediction block of the image block to be processed contains a prediction value of the target sample; and a calculation module configured to perform a weighting calculation of one or more recovered values of one or more reference samples and a prediction value of the target sample to update the prediction value of the target sample, in which the reference sample has a given spatial position relationship with the target sample, wherein the parsing module is further configured to parse the bit stream to obtain a prediction mode corresponding to the image block to be processed and determine that the prediction mode is a merge mode and/or an advanced inter-motion vector prediction (inter-AMVP) mode ). 17. The apparatus of claim 16, wherein the one or more reference samples comprise: a reconstructed sample that has the same horizontal coordinate as the target sample and has a given vertical coordinate difference with the target sample, or a reconstructed sample that has the same vertical coordinate as the target sample and has a given horizontal coordinate difference with the target sample , or a reconstructed sample that has the same horizontal coordinate as the target sample and that is near the top side of the image block to be processed, a reconstructed sample that has the same vertical coordinate as the target sample and that is adjacent to the left side of the image block to be processed, the top right reconstructed sample of the image block to be processed, the lower left reconstructed sample of the image block to be processed, or the top left reconstructed sample of the image block to be processed. 18. The device according to any one of paragraphs. 16, 17, wherein the parsing module is further configured to: perform parsing of the bitstream to obtain information specifying the update definition of the image block to be processed; and determine that the update determination indication information is used to indicate to update the prediction block of the image block to be processed. 19. The device according to any one of paragraphs. 16-18, wherein the weighted coefficient of one or more reconstructed values of one or more reference samples differs from the weighted coefficient of the target sample prediction value. 20. The device according to any one of paragraphs. 16-19, wherein one or more reconstructed values of one or more reference samples are obtained based on a neighboring spatial sample using a planar (PLANAR) intra prediction mode. 21. The device according to any one of paragraphs. 16-20, wherein the updated prediction value predQ(xP, yP) is obtained based on w1*predP(xP, yP) and w2*predP1(xP, yP), where (xP, yP) represents the coordinates of the target sample, predP (xP, yP) represents the prediction value of the target sample, predP1 (xP, yP) represents one or more reconstructed values of one or more reference samples, w2 represents the weighted coefficient of one or more reconstructed values of one or more reference samples, w1 represents the weighted coefficient of the prediction value of the target sample, w1 and w2 are given constants, and w1 is not equal to w2. 22. The apparatus of claim 21, wherein the set (w1, w2) of weighted coefficients is (6, 2) or (5, 3). 23. The apparatus of claim 21, wherein the sum of w1 and w2 is 2 to the power of n and n is 2 or 3. 24. A computer storage medium in which the computer storage medium stores computer program code, and when the computer program code is executed on the processor, the processor is configured to perform an inter prediction method according to any one of claims. 1-15.

Images