RU2533852C2 - Method of encoding/decoding multiview video sequence based on adaptive compensation of local illumination mismatches with interframe prediction (versions) - Google Patents

Abstract

FIELD: physics, video.

SUBSTANCE: invention relates to means of encoding and decoding a multiview video sequence. The method comprises excluding from consideration pixels included in the vicinity of the current encoded block and in the vicinity of the reference block, the pixels being estimated to be unreliable for computation of parameters for changing illumination according to a given criterion; determining numerical ratios between pixels of the reference block, reliable pixels located in the vicinity of the current encoded block and reliable pixels located in the vicinity of the reference block; considering initial values; computing illumination correction parameter values based on the determined numerical ratios; estimating a change in values of pixels of the reference block; when the change is estimated to be substantial, the values of the pixels of the reference block are corrected using the computed correction parameters.

EFFECT: high efficiency of interframe prediction owing to compensation for illumination mismatches between a reference block and a current encoded block based on pixels located in the vicinity of said blocks.

71 cl, 11 dwg

Description

The invention relates to technologies for processing digital signals in a computer system, and more particularly to methods for correcting differences in brightness that may occur between frames of a multi-angle video sequence. In particular, the present invention can be used in the encoding and decoding of multi-angle video sequences.

One of the methods known from the prior art used for coding multi-angle video sequences is to use frames belonging to neighboring views (angles), as well as frames synthesized using frames of neighboring views (angles) and depth maps. Such frames act as reference frames during prediction coding [1]. This eliminates the possible displacement of the object in the current frame relative to one of the reference frames. Under the displacement can be understood the movement of the object or the difference in the position of the object between the current encoded frame and frames belonging to neighboring views (angles), or a synthesized frame. The purpose of eliminating this bias is to minimize the inter-frame difference. The resulting inter-frame difference is then encoded (for example, by applying decorrelation transform, quantization, and entropy coding) and placed in the output bitstream.

Possible differences in the parameters of the cameras used for shooting multi-angle video sequences, as well as the difference in the luminous flux coming from the objects to the cameras, lead to local differences in brightness between frames belonging to different angles. The indicated differences in brightness also affect the characteristics of the synthesized frames. This can lead to an increase in the absolute values of the interframe difference, which negatively affects the coding efficiency.

To solve the above problem, the H.264 standard [2] uses a weighted forecast, which was originally intended for efficient coding of single-species (single-view) video sequences in which there are effects of smooth input and output of images, flicker or scene change. Weighted prediction eliminates the difference in brightness between the encoded frame and reference frames at the macroblock level. In this case, the same values of the weighting coefficients are used for all macroblocks belonging to the same layer. Weights can be determined during the encoding process and stored in the output bitstream (“explicit” weighted forecast) or calculated in the encoding / decoding process (“implicit” weighted forecast). However, in the case of multi-angle sequences, where local changes in brightness and / or contrast can be observed, this method may be ineffective.

Another approach to solving this problem is adaptive block correction of differences in brightness [3]. One of the methods that implements this approach is the method of one-step affine brightness correction for multi-angle video sequences (Multiview One-Step Affine Illumination Compensation - MOSAIC) [4, 5]. The specified method involves a combination of block correction of differences in brightness with the interframe prediction modes described in [2]. In the process of such coding, for each macroblock, the average pixel values of the current encoded block and the candidate reference block are calculated. For these blocks, modified blocks are formed by subtracting the average value for each pixel of the block. Then, for the obtained blocks, the Mean-Removed Sum of Absolute Difference (MRSAD) is calculated. The result of inter-frame prediction is the relative coordinates of the reference block (offset vector), which give the minimum value of the encoding, as well as the difference between the modified encoded block and the modified reference block. In this case, the calculation of the encoding cost is based on the calculated MRSAD value and an estimate of the bit cost of transmitting additional information necessary for subsequent decoding. In addition to the displacement vector, additional information includes the difference between the average values of the current and reference blocks. This difference is referred to as DVIC (Difference Value of Illumination Compensation) and is a brightness correction parameter. The DVIC value is differential encoded and placed in the output bitstream. It should be noted that in the case of the "P Skip" mode, the DVIC value is determined based on the DVIC values of neighboring macroblocks that were already encoded at the time of encoding the current macroblock. Thus, the above method does not completely eliminate the need for explicit transmission of additional information necessary for subsequent decoding.

The parameters necessary for correcting the difference in brightness can be obtained by analyzing the reconstructed (encoded and then decoded) frame regions. This reduces the amount of additional information that must be encoded in the output bitstream. The indicated approach was implemented in the method of weighted prediction using neighboring pixels (WPNP - Weighted Prediction using Neighboring Pixels) [6]. This method uses the pixel values of the encoded frame adjacent to the current encoded block and the pixel values of the reference frame adjacent to the reference block for pixel-by-pixel estimation of brightness changes. In this case, the brightness changes for the selected two neighboring pixels are multiplied by weighting factors and added up, forming an estimate of the brightness and contrast changes between the individual pixels of the current and reference blocks. It should be noted that weights are calculated separately for each pixel position of the encoded block. The values of the weighting coefficients are determined based on the mutual distance between the pixel of the encoded block and the selected neighboring pixels. The accuracy of the estimation is influenced by the fact that a change in the brightness of pixels adjacent to the encoded and reference blocks may differ from a change in the brightness of pixels belonging directly to the encoded and reference blocks, and also that the method according to [6] takes into account only the mutual spatial position of pixels, but does not take into account the ratio of intensities of the adjusted pixel and the pixel from the neighborhood.

Another option that implements the approach associated with assessing the parameters of changing the brightness and contrast by analyzing the reconstructed (encoded, and then decoded) areas of the frames is described in US patent application 2011/0286678 [7]. The encoding method for multi-angle video sequences described in the application includes the correction of the brightness difference in the prediction encoding process. The brightness correction parameters are estimated based on the assessment of changes in the brightness of areas adjacent to the coded and reference blocks of the areas. Since these areas are available both at the coding stage and at the decoding stage, there is no need to explicitly transmit correction parameters in the output bitstream. The resulting parameters are used to correct the reference block. The reliability of the estimation of brightness variation parameters is determined by adjusting the brightness for the region of the reference frame adjacent to the reference block, and comparing the obtained corrected region with the reconstructed (encoded and then decoded) region of the encoded frame adjacent to the current encoded block. The disadvantage of this method is that the calculation of the correction parameters is performed on the basis of all the pixels of the selected adjacent areas, while it does not exclude from consideration individual pixels that can adversely affect the obtained correction parameters, but only estimates the reliability of the correction as a whole, based on which then a decision is made on whether it makes sense to perform the correction or not. This can lead to the elimination of the correction stage as a whole, or to erroneous correction, thereby reducing the efficiency of inter-frame coding.

Closest to the claimed invention is the method described in US patent application 2008/0304760 [8]. The specified method is implemented by a computer system and is aimed at correcting changes in brightness and contrast for the reference block. Moreover, this method includes the following steps: obtaining the restored pixel values adjacent to the current block being encoded, and the restored pixel values adjacent to the reference block, as input information; predicting average values for the current encoded and reference blocks based on the restored pixel values adjacent to the current encoded block and the restored pixel values adjacent to the reference block; determining brightness correction parameters for the reference block based on the predicted average pixel value of the current encoded block, the predicted average value of the reference block and pixel values of the current encoded block and the reference block; and performing luminance correction for the reference block using a previously determined luminance correction parameter.

The disadvantage of the prototype [8] is as follows. The reconstructed values of pixels adjacent to the current encoded block and the reference block are used solely to predict average values. This restriction does not allow the use of information contained in neighboring pixels. In addition, there is no analysis of the relationship between the pixel values of the reference block and the values of pixels adjacent to the reference block. Thus, possible differences in the brightness correction correction parameters between the blocks under consideration and the regions adjacent to the blocks under consideration are not taken into account. This can lead to a decrease in the reliability of the procedure for correcting differences in brightness and contrast, which will negatively affect the encoding efficiency. In addition, the method does not allow to take into account the presence of pixels in adjacent areas, the values of which for various reasons, for example, due to introduced distortions during compression with losses, turn out to be very different from the total set of pixels in adjacent areas. In view of this, the accuracy of the final estimation of the brightness correction parameters can be significantly reduced. Another disadvantage of the prototype is that the correction parameters are always transmitted in the output bitstream, which, in general, can degrade the compression efficiency.

The problem to which the claimed invention is directed is to develop a method implemented by a computer system that can improve the efficiency of inter-frame prediction when encoding a multi-angle video sequence in comparison with the prototype [8].

The technical result is achieved by developing a group of methods for computer processing digital signals united by a single concept, based on an improved method of adaptive local compensation of brightness differences between the reference block and the current encoded block based on pixels from the neighborhoods of the reference and encoded blocks, the values of which are estimated as reliable. The method of adaptive local compensation includes the following operations:

- get the pixel values of the current encoded block belonging to the encoded frame, and the pixel values of the reference block belonging to the reference frame;

- receive the restored (encoded and then decoded) pixel values adjacent to the current block of the encoded frame, and pixel values adjacent to the reference block of the reference frame; wherein the resulting pixels are selected from one or more spatial areas, also called neighborhoods, each of which is characterized by a predetermined spatial proximity with respect to the current block of the encoded frame and the current reference block of the reference frame; areas that are adjacent to the current block of the encoded frame and the current reference block of the reference frame are selected based on the type and size of the spatial transform used subsequently to encode the residual signal, based on the sizes of neighboring blocks already encoded, as well as their logical connections with the current coded block;

- exclude from consideration pixels belonging to both the vicinity of the current encoded block and the vicinity of the reference block of the reference frame, which are estimated as invalid for calculating the parameters of the brightness change according to a predetermined at least one criterion;

- determine the numerical relationship between the pixels of the reference block, valid pixels from the vicinity of the current encoded block and reliable pixels from the neighborhood of the reference block;

- calculate the values of the brightness correction parameters based on the found numerical relationships, taking into account the initial values;

- evaluate changes in the pixel values of the reference block; if the change is estimated as significant, the correction of the pixel values of the reference block is performed using the found correction parameters.

In this context, logical relationships are understood as certain objectively existing dependencies between the current block and neighboring blocks, specified, for example, by the encoding method. In this case, such a connection can be the union of neighboring blocks and the current encoded block together into a single coding element for which a common displacement vector is specified. In addition, an estimate of the similarity of a plurality of pixels adjacent to the encoded block and a plurality of pixels adjacent to the reference block is calculated. The value of the calculated pixel similarity score can be used as an additional condition when deciding whether to compensate for differences in the brightness of the pixels of the reference block.

Thus, the claimed invention consists in the application of a more reliable procedure for evaluating the compensation parameters for brightness differences between the reference and encoded blocks, the procedure for correcting the pixel values of the reference block, as well as in the use of an effective block encoding mode based on a small set of additional service data necessary for implementation. More specifically, the essence of the claimed invention consists in using more data to estimate the brightness variation parameters, as well as taking into account existing relationships between the pixel values of the reference block, the restored pixel values adjacent to the reference block, and the restored pixel values adjacent to the encoded block. In addition, it is planned to use the criteria for similarity of areas for detecting cases when the correction may be ineffective, new methods for identifying pixels that are very different from the set of pixels in adjacent areas, on the basis of which the parameters are estimated, and methods for eliminating very different pixel values from further consideration.

In particular, in one of the implementations of the proposed method, an analysis is made of the relationships between the pixel values of the reference block and the pixel values adjacent to the reference block, as well as the ratios between the restored pixel values adjacent to the current encoded block and pixel values adjacent to in relation to the support block.

When implementing the method, it is also envisaged to use improved methods of encoding and decoding multi-angle video sequences, and such methods are based on the use of brightness compensation, which allows to increase the compression efficiency due to the fact that when evaluating changes in brightness, pixel values are used that are available both for encoding and decoding . In this case, the brightness correction parameters can be accurately restored without the need to transmit any additional data in the output bitstream.

In another implementation of the proposed method, when calculating an estimate of the brightness variation parameters, it is proposed to exclude from consideration pixels that belong to already decoded and reconstructed regions of the reference and encoded frames and whose mutual differences, calculated numerically, exceed the pixel differences of the corresponding adjacent regions according to a predetermined criterion. Such pixels are hereinafter referred to as invalid.

In one embodiment of the claimed invention, a modification of the above method is proposed, which consists in the fact that the process of determining the relationship between the pixels of the current encoded frame and the reference frame, as well as determining the brightness correction parameters, includes the following operations:

- calculate the statistical characteristics for the restored values of pixels adjacent to the current encoded block, the statistical characteristics for the pixels of the reference block and the statistical characteristics for pixels adjacent to the reference block;

- determine the relationship between the statistical characteristics for the pixels of the reference block and the statistical characteristics for the restored values of pixels adjacent to the reference block;

- calculate the value of the statistical characteristics for the current encoded block based on the calculated statistical characteristics and the relationships between them produce:

- determining a brightness correction correction parameter to compensate for brightness differences between the reference and the current encoded blocks based on the found estimate of the statistical characteristic for the current block and the statistical characteristic of the reference block.

Another modification of the claimed invention is that the method for correcting the brightness of the reference block in the process of coding a multi-angle video sequence includes:

- obtaining pixel values of the current block of the encoded frame and pixel values of the reference block of the reference frame;

- obtaining restored (encoded and then decoded) pixel values adjacent to the current block being encoded, and pixel values adjacent to the reference block;

пикселей, соседних по отношению к текущему кодируемому блоку, k=0, …, N-1, N - число пикселей, соседних по отношению к текущему кодируемому и опорному блокам;- calculating a first estimate estD _{i, j} for each position (i, j) of the pixel in the reference block; moreover, the first estimate estD _{i, j} is a linear combination of the restored values

pixels adjacent to the current encoded block, k = 0, ..., N-1, N is the number of pixels adjacent to the current encoded and reference blocks;

пикселей, соседних по отношению к опорному блоку, k=0, …, N-1;- calculating a second estimate estR _{i, j} for each position (i, j) of the pixel in the reference block; moreover, the second estimate estR _{i, j} is a linear combination of values

pixels adjacent to the reference block, k = 0, ..., N-1;

- determination of the correction parameters of the brightness change for the correction of each pixel in the reference block; wherein the determination of these parameters is based on the value of the first estimate estD _{i, j, the} value of the second estimate estR _{i, j} , and also on the values of R _{i, j} pixels of the reference block;

- performing correction of brightness changes for each pixel in the reference block, using the brightness and contrast correction parameters found in the previous step.

According to another modification of the claimed invention, the method provides that the calculation of the first and second estimates for each pixel position in the reference block and determination of brightness correction parameters for each pixel position in the reference block includes:

- calculation of the first estimate estD _{i, j} as

,

,

- calculation of the second estimate estR _{i, j} as

,

,

, k=0, …, N-1, восстановленные значения пикселей, соседних по отношению к текущему кодируемому блоку,

, k=0, …, N-1 - значения пикселей, соседних по отношению к опорному блоку, N - это число пикселей, соседних по отношению к текущему кодируемому и опорному блокам; начальное значение BaseR для первой оценки и начальное значение BaseD для второй оценки заданы таким образом, что BaseR и BaseD либо оба неотрицательные, либо оба неположительные числа;where W _k (i, j), k = 0, ..., N-1 - weighting coefficients depending on the intensity of the pixel of the reference block at position (i, j) and the intensity of the pixel with index k = 0, ..., N-1 from a predetermined nearby neighborhood of the support block,

, k = 0, ..., N-1, restored values of pixels adjacent to the current block being encoded,

, k = 0, ..., N-1 are the values of pixels adjacent to the reference block, N is the number of pixels adjacent to the current encoded and reference blocks; the initial BaseR value for the first evaluation and the initial BaseD value for the second evaluation are set such that BaseR and BaseD are either both non-negative or both non-positive numbers;

, если вторая оценка estR_i,j не равна нулю. В противном случае α_i,j полагается равным 1;- determination of correction parameters for brightness changes for each position (i, j) of the pixel in the reference block; this parameter is defined as

if the second estimate estR _{i, j is} not equal to zero. Otherwise, α _{i, j is set} equal to 1;

- performing correction of the brightness change for the reference block by multiplying the value of each pixel of the reference block R _{i, j} by the corresponding correction parameter α _{i, j} .

The claimed invention also provides that the calculation of the first and second estimates for each pixel position in the reference block and determination of brightness correction parameters for each pixel position in the reference block includes:

- calculation of the first estimate estD _{l, j} as

,

,

, k=0, …, N-1 восстановленные значения пикселей, соседних по отношению к текущему кодируемому блоку, N - это число пикселей, соседних по отношению к текущему кодируемому и опорному блокам;where W _k (i, j), k = 0, ..., N-1 and L _K (i, j), k = 0 ... N-1 are two sets of weights, L _K (i, j), k = 0 ... N-1 depends on the position (i, j) of the pixel in the reference block and the index k = 0, ..., N-1 and

, k = 0, ..., N-1 restored values of pixels adjacent to the current encoded block, N is the number of pixels adjacent to the current encoded and reference blocks;

- calculation of the second estimate as

,

,

, k=0, …, N-1 - значения пикселей, соседних по отношению к опорному блоку; начальное значение BaseR для первой оценки и начальное значение BaseD для второй оценки заданы таким образом, что BaseR и BaseD либо оба неотрицательные, либо оба неположительные числа;where W _k (i, j), k = 0, ..., N-1 and L _K (i, j), k = 0 ... N-1 are two sets of weights, L _K (i, j), k = 0 ... N-1 depends on the position (i, j) of the pixel in the reference block and the index k = 0, ..., N-1 and

, k = 0, ..., N-1 are the values of pixels adjacent to the reference block; the initial BaseR value for the first evaluation and the initial BaseD value for the second evaluation are set such that BaseR and BaseD are either both non-negative or both non-positive numbers;

, если вторая оценка estR_i,j не равна нулю. В противном случае α_i,j и полагается равным 1;- determination of correction parameters for brightness changes for each position (i, j) of the pixel in the reference block; this parameter is defined as

if the second estimate estR _{i, j is} not equal to zero. Otherwise, α _{i, j} and is set equal to 1;

- performing correction of the brightness change for the reference block by multiplying the value of each pixel of the reference block R _{i, j} by the corresponding correction parameter α _{i, j} .

Another modification of the claimed invention provides that the calculation of the first and second estimates for each pixel position in the reference block includes:

- calculation of weighting factors W_k(i, j) , k = 0, ...N-1 for the first estD score_{i, j} and second estR score_{i, j}; for each position (i, j) of the pixel in the reference block, the weight coefficient W_k(i, j) is equal to the non-increasing function of the absolute difference:

,

,

- значение пикселя, соседнего по отношению к опорному блоку; N - число пикселей, соседних по отношению к текущему кодируемому и опорному блокам.which provides an inversely proportional increase / decrease in the value of W _k (i, j) depending on the decrease / increase in the absolute difference. Here R _{i, j} is the pixel value of the reference block;

- value of a pixel adjacent to the reference block; N is the number of pixels adjacent to the current encoded and reference blocks.

Also, another variant of the modification of the claimed invention provides that the calculation of the first and second estimates for each position of the pixel in the reference block includes:

: - calculation of the weighting coefficients L _K (i, j), k = 0, ..., N-1 for the first estimate and the second estimate for each position (i, j) of the pixel in the reference block so that the weighting coefficient L _K (i, j ) is equal to the non-increasing function f of the distance between pixels

:

от корректируемого пикселя R_i,j. Здесь это значение пикселя опорного блока;

- значение пикселя, соседнего по отношению к опорному блоку; N - число пикселей, соседних по отношению к текущему кодируемому и опорному блокам.which provides an inversely proportional increase / decrease in the value of L _K (i, j) depending on the approximation or removal of a pixel

from the corrected pixel R _{i, j} . Here is the pixel value of the reference block;

- value of a pixel adjacent to the reference block; N is the number of pixels adjacent to the current encoded and reference blocks.

In another embodiment of the claimed invention, a modification of the above method is proposed, which provides that the calculation of the first and second estimates for each pixel position in the reference block includes:

- calculation of weight coefficients L _K (i, j), k = 0, ..., N-1 for the first estimate estD _{i, j} and the second estimate estR _{i, j} ; for each position (i, j) of the pixel in the reference block, the weight coefficient is equal to a non-increasing function of the absolute difference:

,

,

, где Thr - предопределенное пороговое значение; иначе w_k(i, j). Здесь R_i,j - значение пикселя опорного блока;

- значение пикселя, соседнего по отношению к опорному блоку; N - число пикселей, соседних по отношению к текущему кодируемому и опорному блокам.which provides an inversely proportional increase / decrease in the value of W _k (i, j) depending on the decrease / increase in the absolute difference in the case

where Thr is a predetermined threshold value; otherwise w _k (i, j). Here R _{i, j} is the pixel value of the reference block;

- value of a pixel adjacent to the reference block; N is the number of pixels adjacent to the current encoded and reference blocks.

When implementing the claimed invention, it makes sense to apply another modification of the above method, which provides that the calculation of the first and second estimates for each pixel position in the reference block includes:

- calculation of weighting factors W _k (i, j), k = 0, ..., N-1 for the first estimate estD _{i, j} and the second estimate estR _{i, j} ; for each position (i, j) of the pixel in the reference block, the weight coefficient W _k (i, j) is equal to a non-increasing function of the absolute difference:

,

,

и

, которые оценены как надежные путем проверки условия

, где

- значение пикселя, соседнего по отношению к текущему кодируемому блоку, Thr1 - первый предопределенный порог; кроме того, вычисление весовых коэффициентов подобным образом выполняется, если

, где thr2 - второй предопределенный порог; иначе W_k(i, j)=0. Здесь R_i,j - значение пикселя опорного блока;

- значение пикселя, соседнего по отношению к опорному блоку; N - число пикселей, соседних по отношению к текущему кодируемому и опорному блокам.which provides an inversely proportional increase / decrease in the value of W _k (i, j) depending on the decrease / increase in the absolute difference; however, the calculation of weights in a similar way is done only for pixels

and

that are rated as reliable by checking the conditions

where

- the value of a pixel adjacent to the current encoded block, Thr1 is the first predetermined threshold; in addition, the calculation of weights in a similar manner is performed if

where thr2 is the second predetermined threshold; otherwise, W _k (i, j) = 0. Here R _{i, j} is the pixel value of the reference block;

- value of a pixel adjacent to the reference block; N is the number of pixels adjacent to the current encoded and reference blocks.

According to another embodiment of the claimed invention, a modification of the above method is proposed, which provides that the calculation of the first and second estimates for each pixel position in the reference block includes:

, где С1, С2, С3 - численные параметры, задающие нелинейную зависимость весового коэффициента от величины A_k(i, j) и A_k(i, j), где определяется как

, R_i,j - значение пикселя опорного блока,

- значение пикселя, соседнего по отношению к опорному блоку, в случае

, где Thr - предопределенный порог; иначе W_k(i, j)=0.- calculation of weighting factors W _k (i, j), k = 0, ..., N-1 for the first estimate estD _{i, j} and the second estimate estR _{i, j} ; for each pixel position in the reference block, the weight coefficient W _k (i, j) is defined as:

, where C1, C2, C3 are numerical parameters defining a nonlinear dependence of the weight coefficient on the quantities A _k (i, j) and A _k (i, j), where it is defined as

, R _{i, j} is the pixel value of the reference block,

is the value of a pixel adjacent to the reference block, in the case

where Thr is a predetermined threshold; otherwise, W _k (i, j) = 0.

, соседними по отношению к опорному блоку. Тем самым может быть учитывается различное влияние пикселей, соседних с текущим блоком и соседних с опорным блоком, и участвующих в вычислении параметров коррекции в зависимости от расстояния до корректируемого пикселя.A further development of the proposed variant of the claimed invention lies in the fact that the parameters determining the dependence of the weight coefficients W _k (i, j), in particular, the parameters C1, C2 and C3 can adaptively vary depending on the distance between the pixel R _{i, j} in the reference block and pixels

adjacent to the support block. Thus, the different influence of pixels adjacent to the current block and neighboring to the reference block and participating in the calculation of correction parameters depending on the distance to the pixel being corrected can be taken into account.

As an alternative to the implementation of the claimed invention, a modification of the above method is proposed, which provides that the calculation of the first and second estimates for each pixel position in the reference block includes:

, где C1, C2, C3 - численные параметры, задающие нелинейную зависимость весового коэффициента от величины A_k(i, j), которая определяется как

, где R_i,j - значение пикселя опорного блока,

- значение пикселя, соседнего по отношению к опорному блоку, а вычисление весовых коэффициентов подобным образом производится только для пикселей

и

, которые оценены как надежные путем проверки условия

, где

, k=0,…, N-1 - значение пикселя, соседнего по отношению к текущему кодируемому блоку, rhr1 - первый предопределенный порог; кроме того, вычисление весовых коэффициентов подобным образом выполняется, если

, где Thr-2 - второй предопределенный порог; иначе W_k(i, j)=0.- calculation of weight coefficients W _k (i, j), k = 0, ..., N-1 for the first estimate estD _{i, j} and the second estimate estR _{i, j} , for each position W _k (i, j) of the pixel in the reference block weight coefficient w _k (i, j) is equal to

, where C1, C2, C3 are numerical parameters defining a nonlinear dependence of the weight coefficient on the quantity A _k (i, j), which is defined as

Where R _{i, j} - pixel value of the reference block,

- the value of a pixel adjacent to the reference block, and the calculation of weighting coefficients in this way is performed only for pixels

and

that are rated as reliable by checking the conditions

where

, k = 0, ..., N-1 is the value of a pixel adjacent to the current block being encoded, rhr1 is the first predetermined threshold; in addition, the calculation of weights in a similar manner is performed if

where Thr-2 is the second predetermined threshold; otherwise, W _k (i, j) = 0.

The claimed invention provides for the implementation of a method in which to reduce the computational cost associated with the compensation of the pixels of the reference block to identify cases when the first estimate estD_{i, j} and second estR score_{i, j} such that the correction parameter α_{i, j} close to 1, the condition is checked: EstR_{i, j}+ Δ1 >> Qbits_one== EstD_{i, j}+ Δ1 >> Qbits_onewhere Qbits_one and Δ1 are non-negative integers, and   denotes the operation of integer rounding, and if the condition is true, then the correction parameter is set to 1.

To improve the accuracy of the method of compensating for local differences in brightness, at which the computational costs associated with pixel compensation of the reference block are reduced in case the first estimate is estD_{i, j} and second estR score_{i, j} such that the correction parameter α_{i, j} close to 1, the condition can be checked ( | EstD_{i, j}-EstR_{i, j}|  + Δ2 >> Qbits₂== 0, where Obits₂ and Δ2 are non-negative integers, and   denotes the operation of integer rounding, and if the condition is true, then the correction parameter is set to 1. It also makes sense to consider a more general criterion that allows us to determine the situation where the correction parameter is close to 1, which implies checking the following inequality: Δ3 × EstR_{i, j}<EstD_{i, j}<Δ4 × EstR_{i, j}, where Δ3, Δ4 are predefined non-negative numbers, and in case of its execution, the correction parameter is set to 1.

The invention also provides an assessment of the similarity of the restored pixels adjacent to the reference block, and the restored pixels adjacent to the encoded block to make a possible decision on the application of correction in the case of strong similarity. In particular, to assess the similarity of the restored pixels adjacent to the reference block, and the restored pixels adjacent to the encoded block, the claimed invention provides:

, соседние с опорным блоком, и все уже восстановленные пиксели

, k=0, …, N-1, соседние с кодируемым блоком, образуя совместно пары

, однозначно задаваемые пространственным положением пикселей

и

, k=0, …, N-1 в соответствующих областях, которое определяется значением номера пикселя k, группируются в М непересекающихся групп G0…GM-1, таких что пара

, k=0…N-1∈{G₀∩…G₁∩…∩G_M-1}. Пары пикселей, принадлежащих группе Gi, обозначаются так же, как

;- all pixels already recovered

adjacent to the reference block, and all pixels already restored

, k = 0, ..., N-1, adjacent to the encoded block, forming together pairs

uniquely determined by the spatial position of the pixels

 and

, k = 0, ..., N-1 in the corresponding areas, which is determined by the value of the pixel number k, are grouped in M disjoint groups G0 ... GM-1, such that the pair

, k = 0 ... N-1∈ {G₀∩ ... G_one∩ ... ∩G_M-1}. Pairs of pixels belonging to the group Gi are denoted in the same way as

;

и

и которая не зависит от различий средних значений всех пикселей

и средних значений всех пикселей

каждой группы Gi;- for each group Gi, a metric is calculated that is based on taking into account the differences between

 and

 and which does not depend on differences in the average values of all pixels

 and average values of all pixels

 each group Gi;

и средних значений всех пикселей

каждой группы Gi;- for each group Gi, a metric is calculated based on differences in the average values of all pixels

and average values of all pixels

each group Gi;

и

по каждой из групп и которая не зависит от различий средних значений всех пикселей

и средних значений всех пикселей

каждой группы Gi;- the general metric, denoted by MR_Norm, is calculated for the groups G0 ... GM-1, which is based on the consideration of pixel differences in pixels

 and

 for each of the groups and which does not depend on differences in the average values of all pixels

 and average values of all pixels

 each group Gi;

и средних значений всех пикселей

каждой группы Gi;- the general metric, denoted by M_Norm, is calculated for groups G0 ... GM-1, which is based on the difference in the average values of all pixels

and average values of all pixels

each group Gi;

- the values of the calculated MR_Norm and M_Norm metrics are compared with the predetermined threshold values, and if MR_Norm is greater than the first specified threshold value or M_Norm is less than the second specified threshold value, the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block considered weak; otherwise, the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is considered strong;

- if the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is weak, correction may not be applied;

- if the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is strong, the correction is performed unconditionally.

Further development of the method provides for the following:

и

и которая не зависит от различий средних значений всех пикселей

и средних значений всех пикселей

каждой группы Gi по формуле: - for each group Gi, the metric MR_Norn (G_l), which is based on pixel differences

 and

 and which does not depend on differences in the average values of all pixels

 and average values of all pixels

 each Gi group according to the formula:

,

,

where the value of the power exponent P1 is determined experimentally;

и средних значений всех пикселей

каждой группы Gi по формуле:- for each group Gi, the metric M_NormG _{l is} calculated, based on the differences in the average values of all pixels

and average values of all pixels

each Gi group according to the formula:

,

,

where the value of the power exponent P2 is determined experimentally;

и

по каждой из групп за исключением различий средних значений всех пикселей

и средних значений всех пикселей

каждой группы Gi по формуле:the general metric, denoted by MR_Norm, is calculated for groups G0 ... GM-1, which is based on the consideration of pixel differences

 and

 for each of the groups, with the exception of differences in the average values of all pixels

 and average values of all pixels

 each Gi group according to the formula:

;

;

и средних значений всех пикселей

каждой группы Gi по формуле:the general metric, denoted by M_Norm, is calculated for groups G0 ... GM-1, which is based on the difference in the average values of all pixels

and average values of all pixels

each Gi group according to the formula:

;

;

- the values of the calculated MR_Norm and M_Norm metrics are compared with the predetermined threshold values, and if MR_Norm is greater than the first specified threshold value or M_Norm is less than the second specified threshold value, the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block considered weak; otherwise, the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is considered strong;

- if the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is weak, the correction may not be applied;

- if the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is strong, correction is required.

и

, которые принадлежат уже декодированным и восстановленным областям опорного и кодируемого кадров и которые существенно отличаются друг от друга, для каждого k=0…N-1 выполняют проверку условия

, где Thr1 неотрицательное пороговое значение, задаваемое наперед.To improve the accuracy of assessing differences in the brightness of the pixels of the reference block in comparison with the encoded block, the claimed invention provides that to exclude from the calculation the values of the received pixels

and

which belong to the already decoded and reconstructed regions of the reference and encoded frames and which differ significantly from each other, for each k = 0 ... N-1, they check the condition

where Thr1 is a non-negative threshold value set in advance.

и

, которые принадлежат уже декодированным и восстановленным областям опорного и кодируемого кадров и которые существенно отличаются от общей выборки (совокупности) пикселей в указанных областях выполняют следующие шаги:The development of the method suggests that in order to further improve the accuracy of assessing the differences in the brightness of the pixels of the reference block in comparison with the encoded block, the claimed invention provides that, to exclude the values of the obtained pixels from the calculations

and

which belong to the already decoded and reconstructed areas of the reference and encoded frames and which differ significantly from the total sample (set) of pixels in these areas perform the following steps:

таким образом, что для всех пикселей

, удовлетворяющих условию

и

, пиксели

, которые соответствуют пикселям

исходя из их порядкового номера k=0…N-1 и пространственного положения в опорном и кодируемом кадрах, группируются в группы, обозначаемые B(LR_i, LR_i+1):- group pixels

such that for all pixels

satisfying the condition

and

pixels

that match the pixels

based on their serial number k = 0 ... N-1 and the spatial position in the reference and encoded frames, are grouped into groups denoted by B (LR _i , LR _{i + 1} ):

,

,

- in this case, the values LR _i , LR _{i + 1} determine the boundaries of the range defining the group B (LR _i , LR _{i + 1} ) and satisfy the condition LR _i >-1; LR _{i + 1} > R _i . The number of groups B (LR _i , LR _{i + 1} ) N _B is determined experimentally and sets the maximum possible value of the index i used for numbering the quantities (LR _i , LR _{i + 1} ): -1 <LR ₀ <LR ₀ <LR ₀ ... <LR _NB ;

- for each group B (LR _i , LR _{i + 1} ) defined by the values (LR _i , LR _{i + 1} ), the following values are calculated:

группы B(LR_i, LR_i+1) проверяют верность следующих трех условий:- for each pixel

 Group B (LR_i, LR_{i + 1}) verify the following three conditions:

- Condition 1:

;

;

- Condition 2:

;

;

- Condition 3:

; в том случае, если верным является хотя бы одно из проверяемых условий 1…3 для очередного рассматриваемого пикселя

группы B(LR_i, LR_i+1), рассматриваемый пиксель

включается в дальнейшие расчеты параметров коррекции яркости опорного блока.

; in the event that at least one of the checked conditions 1 ... 3 is true for the next pixel in question

group B (LR _i , LR _{i + 1} ), the pixel in question

included in further calculations of the brightness correction parameters of the reference block.

Another embodiment of the claimed invention, aimed at improving the accuracy of assessing differences in the brightness of the pixels of the reference block in comparison with the encoded block, is the following modification.

рассматриваются пиксели

, так что

и

и соответствующие им пиксели

сгруппируются в логические группы, обозначаемые B(LR_i, LR_i+1).For all the above pixel pairs

pixels are considered

, so that

and

and their corresponding pixels

are grouped into logical groups denoted by B (LR _i , LR _{i + 1} ).

,

,

where LR _i , LR _{i + 1} determine the boundaries of the range defining the group B (LR _i , LR _{i + 1} ) and satisfy the condition LRf>-1; LR _{i + 1} > R _i . The number of groups B (LR _i , LR _{i + 1} ) Nb is determined experimentally and sets the maximum possible value of the index i used for numbering the quantities (LR _i , LR _{i + 1} ): -1 <LR ₀ <LRo <LR ₀ ... <LR _NB .

For all pixels T _k belonging to B (LR _i , LR _{i + 1} ), the following values are calculated:

- the average value of Mean (B (LR _i , LR _{i + 1} )) for group B (LR _i , LR _{i + 1} ), determined by the formula:

,

,

where | B (LR_i, LR_{i + 1}) | indicates the number of pixels in group B (LR_i, LR_{i + 1})

is the median value of Med (B (LR _i , LR _{i + 1} )) for group B (LR _i , LR _{i + 1} ), defined as the value of a pixel belonging to group B (LR _i , LR _{i + 1} ) such that the number pixels belonging to group B (LR _i , LR _{i + 1} ) and not exceeding the determined median value in terms of value is equal to the number of pixels belonging to group B (LR _i , LR _{i + 1} ) that are not less than the determined median value of group B ( LR _i , LR _{i + 1} );

- the most likely value of Mod (B (LR _i , LR _{i + 1} )) in group B (LR _i , LR _{i + 1} );

- the average deviation Dev (B (LR _i , LR _{i + 1} )) of pixel values in group B (LR _i , LR _{i + 1} ), which can be calculated using one of the well-known formulas that allow us to estimate the parameters of the spread of pixel values in the group B (LR _i , LR _{i + 1} ), for example, in one of the options, the value Dev (B (LR _i , LR _{i + 1} )) can be calculated by the formula:

,

,

where | B (LR _i , LR _{i + 1} ) | denotes the number of pixels in logical group B (LR _i , LR _{i + 1} ); in another implementation, the value Dev (B (LR _i , LR _{i + 1} )) can be calculated by the formula:

,

,

different from the previous one in that the deviation estimate is unbiased, which is especially important for the case of a relatively small number of elements in group B (LR _i , LR _{i + 1} ); also, for the sake of simplification, the following formula can be applied to estimate the average deviation for group B (LR _i , LR _{i + 1} ):

.

.

Experts can also use other methods of estimating the value of Dev (B (LR _i , LR _j )), which characterizes, in essence, the degree of spread of pixels belonging to group B (LR _i , LR _{i + 1} ).

To increase the reliability of further estimates, it is intended to exclude from further consideration those pixel values of group B (LR _i , LR _j ) for which the absolute value of the difference between the value of an individual pixel and the average value of group B (LR _i , LR _j ) and / or the median value of group B (LR _i , LR _j ) and / or the most probable value for group B (LR _i , LR _j ) is greater than the value Dev (B (LR _i , LR _j )) calculated according to one of the well-known formulas, multiplied by a value greater than 0, which defines the jointly permissible spread in group B (LR _i , LR _j ).

According to another embodiment of the claimed invention, a modification of the aforementioned method is proposed in which it is provided that the positions of the restored pixel values adjacent to the current encoded block and the positions of the pixel values adjacent to the reference block are adaptively determined instead of the corresponding pixels with the predetermined positions .

According to another embodiment of the claimed invention, a modification of the aforementioned method is proposed, which provides that, when retrieving the restored pixel values from the vicinity of the current and reference blocks, thinning is performed according to a predetermined pattern. By a template we mean a lot of spatial positions of pixels, which are determined by the given rules inside the neighborhood of the current and reference blocks. At the same time, not all pixels of the specified neighborhoods are selected, but only a part of them. The values of the selected pixels provide an idea of the values of the intensities of all the pixels (their possible values, distribution, etc.) of the regions under consideration with a given accuracy, while reducing the amount of processed information. For example, thinning out the neighborhood of the current and reference blocks is performed by eliminating pixels, at least one of the coordinates of which is even, which corresponds to thinning twice vertically and horizontally. The above example is illustrative and does not limit the many possible ways of forming a template.

The group of inventions related by a single concept also includes an original method of coding multi-angle video sequences, based on adaptive compensation of local differences in brightness during inter-frame prediction of frames, in which the encoded frame is represented by a set of non-overlapping pixel areas called coding units.

This method includes the following operations:

- selection of the current encoded block belonging to the current encoding unit;

- determination of at least one reference block, which is used to form a prediction block for the current block being encoded and whose pixels are already fully encoded and decoded;

- determination of brightness correction parameters of the reference block;

- brightness correction, which consists in correcting the values of all the pixels of the reference block based on certain brightness correction parameters;

- the formation of the prediction block for the current encoded block using the adjusted reference block;

- encoding the current block using the generated prediction block;

wherein the determination of brightness correction parameters and performing brightness correction includes:

- obtaining the restored (encoded and then decoded) pixel values adjacent to the current block of the encoded frame, and pixel values adjacent to the reference block of the reference frame; while the resulting pixels can be selected from one or more spatial regions, each of which is characterized by a predetermined spatial proximity with respect to the current block of the encoded frame and the current reference block of the reference frame; the selection of these regions that are adjacent to the current block of the encoded frame and the current reference block of the reference frame can be made based on the type and size of the spatial transform used subsequently to encode the interframe difference, already decoded pixels, based on the sizes of neighboring blocks already encoded, and also their logical connections with the current block being encoded (logical connections are understood as certain objectively existing dependencies between the current block and neighboring and blocks asked, for example, the encoding method). In this case, such a connection can be the union of neighboring blocks and the current encoded block together into a single coding element for which a common displacement vector is specified. Additionally, a similarity score is calculated for the set of pixels adjacent to the block being encoded and the set of pixels neighboring for the reference block.

The value of the calculated pixel similarity score can be used as an additional condition when deciding on the use of pixel brightness correction of the reference block;

- an exception from consideration when determining the parameters for changing the brightness of unreliable pixels that belong to already decoded and reconstructed areas of the reference and encoded frames and which differ from the total sample (set) of pixels in these areas according to a predetermined criterion, which is based on an analysis of the distribution of pixel values in the specified areas, calculation of statistical characteristics, as well as a comparison of the values of all tested pixels and statistical characteristics in these areas;

- determination of the numerical relationships between the pixels of the reference block, valid pixels from the vicinity of the current decoded block and reliable pixels from the vicinity of the reference block;

- calculation of the values of the brightness correction parameters based on the found numerical relationships taking into account the initial values of the correction parameters;

- assessment of changes in the pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters.

Within the framework of a single inventive concept, it is also envisaged to use an original method for decoding multi-angle video sequences, based on adaptive compensation of local differences in brightness during inter-frame prediction of frames, in which the decoded frame is a set of non-overlapping areas of pixels called coding units. This method includes:

- selection of the current decoded block belonging to the current coding unit;

- determination of the reference block for the current decoded block;

- determination of correction correction parameters for brightness correction of the found reference block;

- brightness correction, which consists in correcting the values of all the pixels of the reference block based on certain brightness correction parameters;

- generating a prediction block for the current decoded block using the adjusted reference block;

- decoding the current block using the generated prediction block;

wherein the determination of brightness correction parameters and performing brightness correction includes:

- obtaining the restored (encoded and then decoded) pixel values adjacent to the current block of the encoded frame, and pixel values adjacent to the reference block of the reference frame; while the resulting pixels can be selected from one or more spatial regions, each of which is characterized by a predetermined spatial proximity with respect to the current block of the encoded frame and the current reference block of the reference frame; the selection of these regions that are adjacent to the current block of the encoded frame and the current reference block of the reference frame can be made based on the type and size of the spatial transform used subsequently to encode the interframe difference, already decoded pixels, based on the sizes of neighboring blocks already encoded, and also their logical connections with the current block being encoded (logical connections are understood as certain objectively existing dependencies between the current block and neighboring and blocks asked, for example, the encoding method in this case, such a connection may be to combine the neighboring blocks and the current block to be coded together in a single coding unit for which the total displacement vector set).; additionally, an assessment of the similarity of the set of pixels adjacent to the encoded block and the set of pixels adjacent to the reference block is calculated. The value of the calculated pixel similarity score can be used as an additional condition when deciding on the use of pixel brightness correction of the reference block;

- an exception from consideration when determining the parameters for changing the brightness of unreliable pixels that belong to already decoded and reconstructed areas of the reference and encoded frames and which differ from the total sample (set) of pixels in these areas according to a predetermined criterion, which is based on an analysis of the distribution of pixel values in the specified areas, calculation of statistical characteristics, as well as a comparison of the values of all tested pixels and statistical characteristics in these areas;

- determination of the relationship between the pixel values of the reference block and the values of pixels adjacent to the reference block, as well as the ratios between the restored pixel values adjacent to the current encoded block and the pixel values adjacent to the reference block;

- determination of the numerical relationships between the pixels of the reference block, valid pixels from the vicinity of the current decoded block and reliable pixels from the vicinity of the reference block;

- calculation of the values of the brightness correction parameters based on the found numerical relationships taking into account the initial values of the correction parameters;

- assessment of changes in the pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters.

Further, the invention is illustrated with the use of graphic materials.

Figure 1 is a structural diagram of a hybrid encoder multi-angle video sequences and the place of application of the claimed invention.

Figure 2 is a structural diagram of a part of a hybrid video encoder that implements the inventive method, which is part of the predictive coding process.

Figure 3 is a diagram showing the main elements of the encoded reference frame that are involved in calculating the correction of brightness changes for the reference block in accordance with the claimed invention.

4 is a diagram illustrating the concept of a super-block, one of the possible combinations of block sizes included in the super-block, as well as two types of spatial transformations used for decorrelation of difference data.

5 (view 5.1 and view 5.2) are diagrams illustrating the procedure for selecting input data in the current frame during the calculation of brightness correction parameters according to one example of the implementation of the claimed invention, as well as the concept of spatial proximity.

6 is a diagram illustrating a method for correcting brightness changes for a reference block in accordance with one embodiment of the claimed invention.

Fig. 7 is a flowchart illustrating a method for pixel correction of brightness variation for a reference block according to one embodiment of the claimed invention.

Fig. 8 is a diagram explaining a method for correcting a brightness change for a reference block in accordance with another embodiment of the claimed invention.

Fig.9 is a flowchart describing a method of encoding multi-angle video sequences based on the correction of brightness changes according to one example of the implementation of the claimed invention.

Figure 10 is a flowchart describing a method for decoding multi-angle video sequences based on the correction of brightness changes according to one example implementation of the claimed invention.

Figure 1 shows a structural diagram of a hybrid encoder multi-angle video sequences. The input to the hybrid multi-view video sequence encoder 105 includes the original view (encoded view) 101 and the already encoded and then decoded views (views) 102 that are part of the encoded multi-view video data. Already encoded / decoded views 102 and already encoded / decoded sequences of depth maps 103 are used to form a synthesized view (view) for the original view (encoded view) using synthesis procedure 104. The generated synthesized view (angle) is also input to the hybrid encoder 105.

The hybrid encoder 105 contains the following functional blocks used for encoding the original view: the reference frame control unit 106, the inter prediction unit 107, the intraframe prediction unit 108, the inter-frame and intraframe compensation unit 109, the spatial transform unit 110, the speed ratio optimization unit 111 / distortion, entropy encoding unit 112. Detailed information on the mentioned functional blocks is contained in [9]. The inventive method can be implemented within the block 107 inter-frame prediction.

Figure 2 contains a diagram of a portion of a hybrid video encoder that implements the inventive method as part of a prediction encoding functional block. The hybrid encoder includes a subtraction unit 201, a transform and quantization unit 202, an entropy encoding unit 203, an inverse transform and inverse quantization unit 204, a bias compensation and correction unit for brightness / contrast changes 205, a view synthesis unit 206, an addition unit 207 , a block 208 for buffering reference frames and depth maps, a block 209 for predicting compensation and correction parameters, a block 210 for estimating an offset and changing the brightness / contrast, and a block 211 for deciding on a macroblock coding mode. Blocks 201-204, 207-209 and 211 are standard coding blocks that are used in the basic hybrid coding method [9]. Block 206 synthesis of the form (view) is a block characteristic of systems for multi-angle coding, because performs the formation (synthesis) of additional reference frames from already encoded / decoded frames and depth maps.

The inventive method is implemented in blocks 205 and 210. These blocks implement a predictive block coding method, which includes the following steps:

- for the current block of the current encoded frame, a search for the reference block is performed, and the offset vector (DV) is determined by a pair of numbers (i, j), which usually determines the cumulative optimum between the bit costs of encoding the offset vector as such and the spacing between the encoded block and adjusted reference block, which can, for example, be calculated by the formula:

where I (m, n) is the pixel brightness value with coordinates (m, n) inside the current block. The size of the current encoded block is H × W. (i, j) defines a displacement vector that points to the reference block R within the predefined search area, Ψ (x) is a function that corrects for possible differences in brightness and contrast between the current block and the reference block. A power exponent P greater than 0 usually takes one of the values 1 or 2. The described method is implemented in block 210. The brightness correction correction parameters found, together with the obtained DV, are transmitted to block 205 and to block 209. In practical implementation, the displacement vector can also be determined at once for the several blocks considered below in order to more compactly present the service information, in this case the minimization criterion will be set for the super-block, which includes several blocks considered further below. In this case, the minimization criterion can be defined as follows:

,

,

where H1, W1 are the sizes of the super block.

The found reference block is converted in accordance with the found brightness correction correction parameters (block 205). After that, block 201 forms a difference block. Then, the difference block is converted using the Discrete Cosine Transform (DCT), quantized (block 202) and encoded by an entropy encoder (block 203). Additional data (SI) necessary for subsequent decoding is also encoded by an entropy encoder (block 203).

Figure 3 schematically shows the main elements of the encoded, reference frames involved in the calculation of the correction of brightness changes for the reference block in accordance with the claimed invention. In accordance with FIG. 3, for the current block 311 (which may be part of the encoded super block 315), the current encoded frame 310, a displacement vector (DV) 320 is determined, which can also be determined for the encoded super block 315. The vector 320 allows uniquely determine the reference block 301 from the reference frame 300 for the current encoded block 311, and the offset vector also uniquely sets the reference super block 305, which corresponds to the encoded super block 315. It should be noted that the reference frame 300, blocks 301, 302, 303 , 304, 305 and restored (per encoded and then decoded) blocks of the current frame 310: 312, 313, 314 are available both during encoding and during decoding.

4, the encoded frame 400 contains within itself a super-block 401, which is characterized in that it can include one or more coded blocks, and for the super-block an offset vector 407 is defined that uniquely determines the reference super-block 408 in the reference frame 406. Super block 401 includes in this case blocks 402, 403, 404, 405, while block 404, like block 402, is encoded using a transform having a smaller size than the transform used in this example for blocks 403 and 405. The choice of super-block sizes, types and times conversion measures are determined by the encoder based on predefined criteria to minimize the cost of encoding information for a given section of the encoded frame. As follows from Figure 4, blocks 402, 403, 404, 405 are logically connected since come in super block 401 as components.

5 (views 5.1 and 5.2) illustrate the relative positioning of the main regions of the current frame 500 considered in the present invention. Thus, the region 501 of the current frame 500 is available during encoding and decoding of the current encoded block 502. Region 501 is sometimes also called a “pattern”. Region 503 is not available during decoding of the current block 502. Since in the claimed invention the calculation of the brightness correction parameters of the reference block is based on the pixels of the current frame, which are located only in region 501, when implementing such correction in the encoder and decoder, there is no need to transmit additional brightness correction parameters in the output bitstream. Some pixels belonging to area 501 are used to calculate brightness correction parameters of the reference block. In Fig. 5 (views 5.1 and 5.2), these pixels are represented by area 505.

The pixels used to calculate the correction parameters can be selected from one or more spatial areas, each of which is characterized by a predetermined spatial proximity with respect to the current block of the encoded frame and the current reference block of the reference frame; so in FIG. 5 (view 5.2), the selected spatial region is region 505, and spatial proximity is determined by the shortest distance between the region and the block boundary, indicated in FIG. 5 (view 5.2) as 506.

The choice of the region can be made based on the type of spatial transformation 202 (Figure 2), which is subsequently used to encode difference pixel information, already decoded pixels, based on the sizes of neighboring blocks already encoded, as well as their logical connections with the current encoded block, specified by the encoding algorithm ; additionally, an assessment is made of the similarity of pixels adjacent to the block being encoded and considered together and pixels adjacent to the reference block and considered together; the value of the calculated pixel similarity score can be used as an additional condition when deciding on the use of pixel brightness correction of the reference block.

One of the main implementations of the claimed invention provides for pixel correction of a change in brightness for a reference block in prediction coding. The key idea is to make a pixel estimate of the brightness correction correction parameter, and the correction is based on the restored pixel values adjacent to the current block, the pixel values of the reference frame, and their mutual similarity. 6 illustrates a specific application of this technique.

.

-пиксели, принадлежащие блокам 602 и 603. Блоки 602 и 603 являются соседними по отношению к опорному блоку 601 и ставятся в соответствие блокам 612 и 613.6, for the current block 611 belonging to the current encoded frame 610, an offset vector (DV) 620 is determined. DV points to the reference block 601 from the reference frame 600. The current block 611, for definiteness, is 4x4 pixels, contains pixels, which are designated as A00 ~ A33. The reference block 601 contains pixels, which are designated as R00 ~ R33. The restored pixel values (blocks 612 and 613, the dimensions of which for definiteness are specified as 4 × 2 and 2 × 4), adjacent to the current encoded block, are indicated as

.

-pixels belonging to blocks 602 and 603. Blocks 602 and 603 are adjacent to the reference block 601 and are mapped to blocks 612 and 613.

For each position (i, j) of the pixel in the reference block 601, the correction of the brightness change is carried out in accordance with the following expression:

_{Ψ (x i, j) =} α i, j · x i, j.

Here, the pixel correction parameter of the brightness change α _{i, j} (in case estR _{i, j is} not equal to 0) is described as:

,

,

where estD _{i, j} is the first estimate for a pixel with coordinates (i, j) in the reference block; estR _{i, j} is the second pixel estimate with the coordinates (i, j) in the reference block. Otherwise, α _{i, j is set} equal to 1.

The block diagram of a method for pixel correction of brightness changes for the reference block is shown in Fig.7. This method includes the following main steps:

1. Obtaining pixel values of blocks 601, 602, 603 from the reference frame 600, block 611 and blocks 612, 613 belonging to the template area of the current encoded frame 610 (operation 701). Further, to evaluate the similarity of the pixels of blocks 602, 603 and the pixels of blocks 612, 613, the following steps are provided:

соседние с опорным блоком (в частности пиксели, входящие в блоки 602 и 603) и все уже восстановленные пиксели

соседние с кодируемым блоком (в частности пиксели, входящие в блоки 612 и 613), образуя совместно пары

, однозначно задаваемые пространственным положением пикселей

и

в соответствующих областях, которое определяется значением номера пикселя к, группируются в M не пересекающихся групп G0…GM-1, таких, что

, k=0…N-1∈{G₀∩…G₁∩…∩G_M-1}. Группой может являться блок пар пикселей с вообще говоря произвольными размерами, например, размерами 2×2 пары пикселей, 4×4 пары пикселей и т.д. в зависимости от фактических размеров исходных блоков 602, 603, 612, 613. Пары пикселей, принадлежащих группе Gi, обозначаются также как

;- all pixels already recovered

adjacent to the reference block (in particular pixels included in blocks 602 and 603) and all pixels already restored

adjacent to the encoded block (in particular, the pixels included in blocks 612 and 613), forming together pairs

uniquely determined by the spatial position of the pixels

and

in the corresponding areas, which is determined by the value of the pixel number k, are grouped in M disjoint groups G0 ... GM-1, such that

, k = 0 ... N-1∈ {G ₀ ∩ ... G ₁ ∩ ... ∩ G _M-1 }. A group can be a block of pairs of pixels with generally speaking arbitrary sizes, for example, sizes 2 × 2 pairs of pixels, 4 × 4 pairs of pixels, etc. depending on the actual sizes of the source blocks 602, 603, 612, 613. Pairs of pixels belonging to the group Gi are also denoted as

;

и

за исключением различия средних значений всех пикселей

и средних значений всех пикселей

каждой группы Gi по формуле:- for each group Gi, the metric MR_Norm (Gi) is calculated, which is based on the consideration of pixel differences

 and

 except for the difference in average values of all pixels

 and average values of all pixels

 each Gi group according to the formula:

,

,

where the value of the power exponent P1 is determined experimentally and for definiteness can be set equal to 1;

и средних значений всех пикселей

каждой группы Gi по формуле:- for each group Gi, the metric M_Norm (G _t ) is calculated based on the differences in the average values of all pixels

and average values of all pixels

each Gi group according to the formula:

,

,

where the value of the power exponent P2 is determined experimentally, and for definiteness can be set equal to 1;

и

по каждой из групп за исключением различий средних значений всех пикселей

и средних значений всех пикселей всех пикселей

каждой группы Gi по формуле:- the general metric, denoted by MR_Norm, is calculated for the groups G0 ... GM-1, which is based on the consideration of pixel differences in pixels

and

for each of the groups with the exception of differences in the average values of all pixels

and average values of all pixels of all pixels

each Gi group according to the formula:

;

;

и средних значений всех пикселей

каждой труппы Gi по формуле:- the general metric, denoted by M_Norm, is calculated for the groups G0 ... GM-1, which is based on the difference in the average values of all pixels

and average values of all pixels

each troupe Gi according to the formula:

;

;

- the values of the calculated MR_Norm and М_Norm metrics are compared with the predetermined threshold values, and if MR_Norm is greater than the first specified threshold value or M_Norm is less than the second specified threshold value, the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block considered weak; otherwise, the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is considered strong;

- if the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is weak, the correction may not be applied;

- if the similarity of the restored pixels adjacent to the reference block and the restored pixels adjacent to the encoded block is strong, correction is necessary.

2. The calculation of the weights W _k (i, j), k = 0, ..., N-1 for each position (i, j) of the pixel in the reference block 601 (operation 702). Weighting factors W _k (i, j) are expressed as follows:

W _k (i, j) = exp (-C1A _k (i, j) ^C2 + C3)

, $$A_k\left(i,j\right)=|\; |\; |R_{i,j}-T_k^R|\; |\; |$$

,

, тем больше его вклад при определении параметра коррекции изменения яркости для опорного блока.where C1, C2, C3 are numerical parameters specifying the nonlinear dependence of the weight coefficient on the quantity A _k (i, j), is determined experimentally. Here N is the total number of pixels in blocks 612, 613 (or 602, 603). It should be noted that the weights reflect the fact that the closer the value of R _{i, j is} closer to $$T_k^R$$

, the greater its contribution in determining the brightness correction correction parameter for the reference block.

3. The calculation of the values estD _{i, j} for each position (i, j) of the pixel in the reference block 601 (operation 703) in accordance with the following expression:

$$est{D}_{i,j}=BaseD+{\displaystyle \sum _{\begin{array}{l}k=0\_N-\mathrm{one}\\ k_i|\; |\; |T_k^R-T_k^D|\; |\; |\le Thr\mathrm{one}and|\; |\; |T_k^R-R_{ij}|\; |\; |\le Thr2\end{array}}{W}_k\left(i,j\right)\times L_k\left(i,j\right)\times T_k^D}$$

, соседних по отношению к текущему кодируемому блоку.Thr1 and Thr2 are predefined threshold values, BaseD is the initial value of the first estimate. Threshold values are used to exclude pixel values adjacent to the reference block, which differ significantly from the values of R _{i, j} , and values $$T_k^D$$

adjacent to the current encoded block.

4. The calculation of values for each position i, j of the pixel in the reference block 601 (operation 704) in accordance with the following expression:

$$est{D}_{i,j}=BaseD+{\displaystyle \sum _{\begin{array}{l}k=0\_N-\mathrm{one}\\ k_i|\; |\; |T_k^R-T_k^D|\; |\; |\le Thr\mathrm{one}and|\; |\; |T_k^R-R_{ij}|\; |\; |\le Thr2\end{array}}{W}_k\left(i,j\right)\times L_k\left(i,j\right)\times T_k^R}$$

The predefined threshold values Thr1 and Thr2 are the same as in the calculation for calculating estD _{i, j} , BaseR is the initial value of the first estimate.

и $$T_k^D$$

, которые принадлежат уже декодированным и восстановленным областям опорного и кодируемого кадров и которые отличаются от общей выборки (совокупности) пикселей в указанных областях, выполняют следующие действия:To improve the accuracy of estimating the value of the correction parameter of the brightness change α _{i, j} , the claimed invention provides that to exclude from consideration the received pixels $$T_k^R$$

and $$T_k^D$$

which belong to the already decoded and reconstructed areas of the reference and encoded frames and which differ from the total sample (set) of pixels in these areas, perform the following actions:

таким образом, что для всех пикселей $$T_k^R$$

, удовлетворяющих условию $$T_k^R>L{R}_i$$

и $$T_k^R<L{R}_{i+1}$$

, пиксели $$T_k^D$$

, которые соответствуют пикселям $$T_k^R$$

исходя из их порядкового номера k=0…N-1 и пространственного положения в опорном и кодируемом кадрах, группируются в группы, обозначаемые B(LR_i, LR_i+1):- group pixels $$T_k^D$$

such that for all pixels $$T_k^R$$

satisfying the condition $$T_k^R>L{R}_i$$

and $$T_k^R<L{R}_{i+\mathrm{one}}$$

pixels $$T_k^D$$

that match the pixels $$T_k^R$$

based on their serial number k = 0 ... N-1 and the spatial position in the reference and encoded frames, are grouped into groups denoted by B (LR _i , LR _{i + 1} ):

$$B\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)=\{\underset{k=0...N-\mathrm{one}}{T_k^D}:T_k^R>L{R}_i\wedge T_k^R<L{R}_{i+\mathrm{one}}\}$$

- in this case, the values LR _b LR _{i +]} determine the boundaries of the range defining the group B (LR _i , LR _{i + 1} ) and satisfy the condition LR _i >-1; LR _{i + 1} > R _i . The number of groups B (LR _i , LR _{i + 1} ) N _B is determined experimentally and sets the maximum possible value of the index i used for numbering the quantities (LR-, LR _{i + 1} ): -1 <LR ₀ <LR ₀ <LR ₀ ... <LR _NB ;

- for each group B (LR _i , LR _{i + 1} ) defined by the values (LR _i , LR _{i + 1} ), the following values are calculated:

$$C\_Plus\left(LRi,LRi+\mathrm{one}\right)={\displaystyle \sum _{T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)}\{\begin{array}{l}\mathrm{one,}{T}_k^R\ge T_k^D+Thr5\\ 0T_k^R<T_k^D+Thr5\end{array}}$$

$$C\_Minus\left(LRi,LRi+\mathrm{one}\right)={\displaystyle \sum _{T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)}\{\begin{array}{l}\mathrm{one,}{T}_k^R\le T_k^D-Thr5\\ 0T_k^R>T_k^D-Thr5\end{array}}$$

группы B(LR_i, LR_i+1) проверяют верность следующих трех условий:- for each pixel $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

groups B (LR _i , LR _{i + 1} ) verify the following three conditions:

Condition 1:

$$|\; |\; |C\_Plus\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)-C\_Minus\left(L{R}_{i,}L{R}_{i+\mathrm{one}}\right)<Thr6|\; |\; |$$

;

Condition 2:

$$C\_Plus\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)-C\_Minus\left(L{R}_{i,}L{R}_{i+\mathrm{one}}\right)>=Thr6AND{T}_k^R\ge T_k^D+Thr5$$

;

Condition 3:

; $$C\_Plus\left(LRi,LRi+\mathrm{one}\right)-C\_Minus\left(LRi,LRi+\mathrm{one}\right)<=-Thr6AND{T}_k^R\le T_k^D-Thr5$$

;

группы B(LR_i, LR_i+1), рассматриваемый пиксель $$T_k^D\in B\left(LRi,LRi+1\right)$$

включается в дальнейшие расчеты параметров коррекции яркости опорного блока.in the event that at least one of the checked conditions 1 ... 3 is true for the next pixel in question $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

group B (LR _i , LR _{i + 1} ), the pixel in question $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

included in further calculations of the brightness correction parameters of the reference block.

5. The calculation of the brightness correction parameter α _{i, j} (operation 705) for each pixel with coordinates (i, j) in the reference block 601 based on the obtained values and estD _{i, j} if estD _{i, j is} not equal to 0. Otherwise, α _{i, j is} assumed to be equal to 1.

It is envisaged that in order to reduce the computational costs associated with compensating the pixels of the reference block if the first estimate estD _{i, j} and the second estimate estR _{i, j} are such that the correction parameter α _{i, j} is close to 1, the condition is checked: EstR _{i, j} + Δ1 >> Qbits ₁ == EstD _{i, j} + Δ1 >> Qbits ₁ , where Qbits ₁ and Δ1 are non - negative integers, and if the condition is true, then the correction parameter is set to 1.

Also, to improve the accuracy of the method for compensating for local differences in brightness, in which the computational costs associated with compensation of pixels of the reference block are reduced if the first estimate estD _{i, j} and the second estimate estR _{i, j} are such that the correction parameter α _{i, j} is close to 1, the condition can be checked ( | EstD _{i, j} - EstR _{i, j} |  + Δ2 >> Qbits ₂ == 0, where Obits ₂ and Δ2 where Qbits ₂ and Δ2 are non-negative integers, and if the condition is true , then the correction parameter is set to 1. It makes sense to consider a more general criterion, which allows to determine the situation spine correction parameter to 1, which means by the following inequality: Δ3 × EstR <EstD <Δ4 × EstR, where Δ3, Δ4 - predetermined non-negative numbers, and in case of performing the task of correction parameter 1.

6. Performing a correction of the brightness change (operation 706) for the reference block 601 based on the use of the calculated parameters α _{i, j} if the pixel similarity of the blocks 602, 603 and 612, 613 is considered strong.

It is worth noting that the claimed invention also provides such a correction of the brightness values of the pixels of the reference block, in which instead of using the intensity of each pixel of the reference block individually, the generalized statistical characteristic of more than one pixel of the reference block can be adjusted. For example, for pixels A00, A01, A10, A11 from block 601, it is possible to calculate the average value for which the correction parameter will be calculated in accordance with the embodiments of the claimed invention described above. Then, the average value of the pixel population A00, A01, A10, A11 will be replaced by the adjusted value. In another implementation, the pixels of the reference block are grouped into one or more groups of pixels on the basis of a clearly defined common feature (for example, proximity of intensities), with the subsequent selection of a common parameter (such as an average value), its correction and the final replacement of the initial value of the common parameter for selected group of pixels on the adjusted.

Another embodiment of the claimed invention is based on the following. Typically, as pixels adjacent to the reference block, a group of pixels adjacent to the reference block is selected. However, the search procedure for the reference block can select such a displacement vector that the pixel values in the indicated group will not be sufficiently similar to the corresponding pixel values adjacent to the current encoded block. Moreover, the values of pixels directly adjacent to the reference block may differ significantly from the pixel values of the reference block. In these cases, correction of changes in brightness and contrast may not be performed correctly.

To solve this problem, in an embodiment of the claimed invention, it is proposed to use the "floating" (relative to the reference block) position of said group of pixels adjacent to the reference block. Fig.8 explains the inventive method in accordance with one embodiment of the claimed invention. 8, at each iteration of the reference block search procedure for the current block 811 of the current encoded frame 810, a displacement vector (DV) 820 is determined. DV points to the reference block 801 of the reference frame 800. The coordinates of the group of pixels of the reference frame (which are formed by the pixels of the blocks 802 and 803) are determined using an additional refinement offset vector 804. The bias refinement vector 804 is the result of an additional bias estimation procedure. In this case, such a displacement vector 804 is determined that gives the minimum value of the penalty function that determines the degree of similarity of blocks 812, 813 and blocks 802, 803, respectively. Such a well-known function as the mean square error, the sum of the absolute differences, the sum of the absolute differences for signals with a zero mean, and so on, can serve as a penalty function. Vector 804 is implicitly determined during the encoding and decoding process without transmitting additional information in the output bitstream.

The logical continuation of the described aspect of the claimed invention is the following modification. Typically, as pixels adjacent to the reference block, a group of pixels adjacent to the reference block is selected. In this case, the search procedure for the reference block can select the displacement vector specified with fractional accuracy, i.e. such that it indicates some intermediate spatial position located between integer pixel positions. In this case, in order to simplify, it is envisaged to select a group of pixels adjacent to the reference block such that their position is specified by one of the closest pixel positions that can be set with less accuracy than the accuracy of the selected displacement vector (for example, instead of quarter pixel accuracy for neighborhoods will use pixel accuracy, which in some implementations may reduce computational complexity).

Figure 9 presents a flowchart that describes a method of encoding multi-angle video sequences based on brightness correction according to one embodiment of the claimed invention. At step 901, the current coded block belonging to the current coding unit is selected. At 902, at least one reference block is determined, which is used to generate a prediction block for the current block being encoded and whose pixels are already fully encoded and decoded. At 903, brightness correction parameters of the reference block are determined. At 904, values of all pixels of the reference block are corrected based on the determined brightness correction parameters. In this case, the determination of the brightness correction parameters and performing the brightness correction includes:

- obtaining the restored (encoded and then decoded) pixel values adjacent to the current block of the encoded frame, and pixel values adjacent to the reference block of the reference frame; while the resulting pixels can be selected from one or more spatial regions, each of which is characterized by a predetermined spatial proximity with respect to the current block of the encoded frame and the current reference block of the reference frame; the selection of these regions that are adjacent to the current block of the encoded frame and the current reference block of the reference frame can be made based on the type and size of the spatial transform used subsequently to encode the interframe difference, already decoded pixels, based on the sizes of neighboring blocks already encoded, and also their logical connections with the current block being encoded (logical connections are understood as certain objectively existing dependencies between the current block and neighboring and blocks asked, for example, the encoding method). In this case, such a connection can be the union of neighboring blocks and the current encoded block together into a single coding element for which a common displacement vector is specified. Additionally, a similarity score is calculated for the set of pixels adjacent to the block being encoded and the set of pixels neighboring for the reference block. The value of the calculated pixel similarity score can be used as an additional condition when deciding on the use of pixel brightness correction of the reference block;

- an exception from consideration when determining the parameters for changing the brightness of unreliable pixels that belong to already decoded and reconstructed areas of the reference and encoded frames and which differ from the total sample (set) of pixels in these areas according to a predetermined criterion, which is based on an analysis of the distribution of pixel values in the specified areas, calculation of statistical characteristics, as well as a comparison of the values of all tested pixels and statistical characteristics in these areas;

- determination of the numerical relationships between the pixels of the reference block, valid pixels from the vicinity of the current decoded block and reliable pixels from the vicinity of the reference block;

- calculation of the values of the brightness correction parameters based on the found numerical relationships taking into account the initial values of the correction parameters;

- assessment of changes in the pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters.

At 905, a prediction block is generated for the current block to be encoded using the adjusted reference block. At 906, the current block is encoded using the generated prediction block.

Figure 10 illustrates a method for decoding multi-angle video sequences based on the correction of brightness changes according to one example implementation of the claimed invention. 10, in step 1001, a current decoded block belonging to the current coding unit is selected. At step 1002, a reference block is determined for the current decoded block. At 1003, change correction parameters are determined to correct the brightness of the found reference block. At 1004, values of all pixels of the reference block are corrected based on the determined brightness correction parameters. In this case, the determination of the brightness correction parameters and performing the brightness correction includes:

- obtaining the restored (encoded and then decoded) pixel values adjacent to the current block of the encoded frame, and pixel values adjacent to the reference block of the reference frame; while the resulting pixels can be selected from one or more spatial regions, each of which is characterized by a predetermined spatial proximity with respect to the current block of the encoded frame and the current reference block of the reference frame; the selection of these areas that are adjacent to the current block of the encoded frame and the current reference block of the reference frame can be made based on the type and size of the spatial transform used subsequently to encode the interframe difference of the already decoded pixels, based on the sizes of neighboring blocks already encoded, and their logical connections with the current block being encoded (logical connections are understood as certain objectively existing dependencies between the current block and the neighboring and blocks specified, for example, by the encoding method. In this case, such a connection can be the combination of neighboring blocks and the current encoded block together into a single encoding element for which a common displacement vector is specified); additionally, an assessment of the similarity of the set of pixels adjacent to the encoded block and the set of pixels adjacent to the reference block is calculated. The value of the calculated pixel similarity score can be used as an additional condition when deciding on the use of pixel brightness correction of the reference block;

- an exception from consideration when determining the parameters for changing the brightness of unreliable pixels that belong to already decoded and reconstructed areas of the reference and encoded frames and which differ from the total sample (set) of pixels in these areas according to a predetermined criterion, which is based on an analysis of the distribution of pixel values in the specified areas, calculation of statistical characteristics, as well as a comparison of the values of all tested pixels and statistical characteristics in these areas;

- determination of the relationship between the pixel values of the reference block and the values of pixels adjacent to the reference block, as well as the ratios between the restored pixel values adjacent to the current encoded block and the pixel values adjacent to the reference block;

- determination of the numerical relationships between the pixels of the reference block, valid pixels from the vicinity of the current decoded block and reliable pixels from the vicinity of the reference block;

- calculation of the values of the brightness correction parameters based on the found numerical relationships taking into account the initial values of the correction parameters;

- assessment of changes in the pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters.

In step 1005, a prediction block is generated for the current decoded block using the adjusted reference block. At 1006, the current block is decoded using the generated prediction block.

The above methods for adaptive compensation of local differences in brightness during inter-frame prediction, as well as methods for encoding and decoding multi-angle video sequences based on adaptive compensation for local differences in brightness, are implemented as a computer system. For example, a computer system may include one or more processors, RAM, memory, and a data input / output unit. In this case, the sequence of operations for each of the above methods is represented by a sequence of processor instructions that are stored in a storage device. Upon receipt of the necessary data on the input / output unit, the processor places in the RAM and / or reads data from the storage device, and then executes a sequence of instructions stored in the storage device, thereby implementing one of the methods in accordance with the present invention.

The embodiments of the claimed invention described above are provided for purposes of illustration only and are not restrictive. The scope of protection of the invention is determined by the attached claims.

References

[1] Yea, S .; Vetro, A., “View Synthesis Prediction for Multiview Video Coding”, Image Communication, ISSN: 0923-5965, Vol.24, Issue 1-2, pp.89-100, January 2009;

[2] ITU-T Rec. H.264. Advanced video coding for generic audiovisual services. 2010.

[3] US Patent 7,924,923. Motion Estimation and Compensation Method and Device Adaptive to Change in Illumination. April 2011.

[4] Y. Lee, J. Hur, Y. Lee, R. Han, S. Cho, N. Hur, J. Kim, J. Kim, P. Lai, A. Ortega, Y.Su, P. Yin and C. Gomila. CE11: Illumination compensation. Joint Video Team (JVT) of ISOAEC MPEG and ITU-T VCEG JVT-U052, Oct. 2006.

[5] J. H. Kim, P. Lai, J. Lopez, A. Ortega, Y.Su, P. Yin, and C. Gomila. New coding tools for illumination and focus mismatch compensation in multiview video coding. IEEE Trans, on Circuits and Systems for Video Technology, vol. 17, no.11, pp. 1519-1535, Nov. 2007.

[6] T. Yamamoto, T. Ikai, "Weighted prediction using neighboring pixels," ITU-T Q.6 / SG16 VCEG, Proposal VCEG-AH19, January 2008.

[7] US Patent Application 2011/0286678. Multi-view Image Coding Method, Multi-view Image Decoding Method, Multi-view Image Coding Device, Multi-view Image Decoding device, Multi-view Image Coding Program, and Multi-view Image Decoding Program. November, 2011.

[8] US patent application 2008/0304760. Method and Apparatus for Illumination Compensation and Method and Apparatus for Encoding and Decoding Image Based on Illumination Compensation. December. 2008.

[9] Richardson I.E. The H.264 Advanced Video Compression Standard. Second Edition. 2010.

[10] Martucci S.A. “Reversible compression of HDTV images using median adaptive prediction and arithmetic coding”, in IEEE Int. Symp on Circuits and Systems, 1990.

Claims (71)

1. The method of adaptive compensation of local differences in brightness during inter-frame prediction of frames in the process of encoding and decoding a multi-angle video sequence, implemented by a computer system and includes the following operations:
- get the pixel values of the current encoded block belonging to the encoded frame, and the pixel values of the reference block belonging to the reference frame;
- get restored, that is, encoded and then decoded, the pixel values from the vicinity of the current encoded block of the encoded frame and the pixel values from the neighborhood of the reference block of the reference frame;
characterized in that:
- exclude from consideration the pixels belonging to the vicinity of the current encoded block and the neighborhood of the reference block of the reference frame, which are estimated as invalid for calculating the parameters of the brightness change according to a predetermined at least one criterion;
- determine the numerical relationship between the pixels of the reference block, valid pixels from the vicinity of the current encoded block and reliable pixels from the neighborhood of the reference block;
- taking into account the initial values, the values of the brightness correction parameters are calculated based on the found numerical ratios;
- evaluate the change in pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters. 2. The method according to claim 1, characterized in that the procedures for determining the ratios for the pixels of the current encoded frame and the reference frame, the determination of the correction parameter of the brightness change, as well as the correction of the pixel values of the reference block include the following steps:
- determine the nearby neighborhood of the current encoded block and the nearby neighborhood of the reference block, so that the pixels belonging to these neighborhoods are already restored;
- calculate at least two numerical estimates estD _{i, j} and estR _{i, j} for each position (i, j) of the pixel in the reference block, based on the restored values $$T_k^D$$

pixels from a specific nearby neighborhood of the current encoded block and values $$T_k^R$$

pixels, from a certain nearby neighborhood of the reference block, where N is the number of pixels from a certain nearby neighborhood of the current encoded block, which is also equal to the number of pixels from a certain nearby neighborhood of the reference block and k = 0, ..., N-1;
- determined based on at least a first estimate estD _{i, j of a} second estimate estR _{i, j of} values R _{i, j of} pixels of the reference block, the restored pixel values $$T_k^D$$

and $$T_k^R$$

a brightness correction correction parameter for each pixel position in the reference block;
- perform the correction of the pixel values of the reference block using the correction parameters. 3. The method according to claim 2, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block and determining the correction parameters of the pixel values of the reference block includes the following operations:
- set the initial value of BaseR for the first evaluation and the initial value of BaseD for the second evaluation so that BaseR and BaseD are either both non-negative or both non-positive numbers;
- calculate the weight coefficients W _k (i, j), k = 0, ..., N-1, depending on the intensity of the pixel of the reference block at position (i, j) and the intensity of the pixel with the index k = 0, ..., N-1 from a predetermined nearby neighborhood of the support block;
- calculate the first estimate as $$est{D}_{i,j}=BaseD+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{W}_k\left(i,j\right)\times T_k^D}$$

;
- calculate the second score as $$est{R}_{i,j}=BaseR+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{W}_k\left(i,j\right)\times T_k^R}$$

;
- if the second estimate estR _{i, j is} not equal to zero, the correction parameter of the value of each pixel of the reference block is determined as $${\alpha }_{i,j}=\frac{est{D}_{i,j}}{est{R}_{i,j}}$$

; otherwise, the correction parameter α _{i, j} is set to 1;
- carry out the correction of the brightness of the reference block by multiplying the value of each pixel of the reference block R _{i, j} by the corresponding correction parameter α _{i, j} if the correction parameter α _{i, j is} not equal to 1. 4. The method according to claim 3, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block and determining the correction parameters for the values of each pixel in the reference block includes the following operations:
- set the initial value of BaseR for the first evaluation and the initial value of BaseD for the second evaluation so that BaseR and BaseD are either both non-negative or both non-positive numbers;
- calculate the weight coefficients W _k (i, j), k = 0, ..., N-1, depending on the intensity of the pixel of the reference block at position (i, j) and the intensity of the pixel with the index k = 0, ..., N-1 from neighborhood of the reference block;
- calculate the second set of weights L _K (i, j), k = 0 ... N-1, depending on the position (i, j) of the pixel in the reference block and the index k = 0, ..., N-1;
- calculate the first estimate as $$est{D}_{i,j}=BaseD+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{W}_k\left(i,j\right)\times L_K\left(i,j\right)\times T_k^D}$$

;
- calculate the second score as $$est{R}_{i,j}=BaseR+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{W}_k\left(i,j\right)\times L_K\left(i,j\right)\times T_k^R}$$

. 5. The method according to claim 3, characterized in that the brightness correction parameter α _{i, j} for each position (i, j) is a non-negative integer, the correction value of each pixel of the reference block is calculated by at least multiplying the value of each pixel by a correction parameter for changing the brightness and bit shift to the right by a predetermined non-negative integer. 6. The method according to claim 3, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block includes the following operations:
- calculate the weight coefficients W _k (i, j), k = 0, ..., N-1 for each position (i, j) of the pixel in the reference block, while the weight coefficient W _k (i, j) is equal to a non-increasing function of the absolute differences $$|\; |\; |R_{i,j}-T_k^R|\; |\; |$$

that provides an inversely proportional increase / decrease W _k (i, j) depending on the decrease / increase in the absolute difference. 7. The method according to claim 4, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block includes the following operations:
- calculate the weight coefficients W _k (i, j), k = 0, ..., N-1 for each position (i, j) of the pixel in the reference block, while the weight coefficient W _k (i, j) is equal to a non-increasing function of the absolute differences $$|\; |\; |R_{i,j}-T_k^R|\; |\; |$$

that provides an inversely proportional increase / decrease W _k (i, j) depending on the decrease / increase in the absolute difference;
- calculate the weight coefficients L _K (i, j), k = 0 ... N-1 for the first estimate estD _{i, j} and for the second estimate estR _{i, j} for each position (i, j) of the pixel in the reference block so that the weight coefficient L _K (i, j) is equal to a non-increasing function f of the distance between pixels $$\Vert R_{i,j},T_K^R\Vert$$

: $$L_K\left(i,j\right)=f\left(\Vert R_{i,j},T_K^R\Vert \right)$$

that provides an inversely proportional increase / decrease in the value of L _K (i, j) depending on the approximation or removal of the pixel $$T_k^R(k=0...,N-\mathrm{one})$$

from the corrected pixel R _{i, j} . 8. The method according to any one of claims 3 or 4, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block before calculating the weight coefficients W _k (i, j), k = 0, ..., N-1 provides for the following operations:
- check the first pixel confidence condition: $$|\; |\; |T_k^R-T_k^D|\; |\; |\le Thr\mathrm{one}$$

where Thr1 is the first predetermined threshold;
- check the second pixel confidence condition: $$|\; |\; |T_k^R-R_{i,j}|\; |\; |\le Thr2$$

where Thr2 is the second predetermined threshold;
- exclude such pixels from further calculations $$T_k^R(k=0...,N-\mathrm{one})$$

and $$T_k^D(k=0...,N-\mathrm{one})$$

, which are evaluated as unreliable for calculation in case of failure to fulfill at least one of the two specified conditions of reliability. 9. The method according to claim 8, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block includes the following operations:
- set the values of the weighting coefficients W _k (i, j), k = 0, ..., N-1 for the first estimate estD _{i, j} and the second estimate estR _{i, j} equal to one for each position (i, j) of the pixel in the reference block for all positions k = 0 ... N-1 such that pixels $$T_k^D$$

and $$T_k^R$$

rated as reliable. 10. The method according to claim 6, characterized in that the procedure for calculating the first and second estimates for each pixel position in the reference block includes the following operations:
- calculate the weight coefficients W _k (i, j), k = 0, ..., N-1 so that the weight coefficient W _k (i, j) is $$W_k(i,j)=e^{\left(-C\mathrm{one}\cdot A_k{\left(i,j\right)}^{C2}+C3\right)}$$

, where C1, C2, C3 are parameters defining a nonlinear dependence of the weight coefficient on a quantity, A _k (i, j) is $$A_k(i,j)=|\; |\; |R_{i,j}-T_k^R|\; |\; |$$

. 11. The method according to claim 10, characterized in that the weight coefficient W _k (i, j) is calculated as:
$$W_K\left(i,j\right)=e^{-C\mathrm{one}\left(\Vert R_{i,j},T_K^R\Vert \right)\times A_K{\left(i,j\right)}^{C2\left(\Vert R_{i,j},T_K^R\Vert \right)}+C3\left(\Vert R_{i,j},T_K^R\Vert \right)}$$

where $$C\mathrm{one}\left(\Vert R_{i,j},T_K^R\Vert \right)$$

, $$C2\left(\Vert R_{i,j},T_K^R\Vert \right)$$

and $$C3\left(\Vert R_{i,j},T_K^R\Vert \right)$$

- parameters functionally dependent on distance $$\Vert R_{i,j},T_K^R\Vert$$

, $$A_k\left(i,j\right)=|\; |\; |R_{i,j}-T_k^R|\; |\; |$$

. 12. The method according to claim 2, characterized in that the position position of a nearby neighborhood of the current encoded block and the position of a nearby neighborhood of the reference block are determined adaptively instead of the positions set initially. 13. The method according to claim 2, characterized in that when calculating at least two numerical estimates estD _{i, j} and estR _{i, j} , only a part of the pixels located in a predetermined nearby neighborhood of the current block and the corresponding part of the pixels are used in a predetermined nearby neighborhood of the reference block, performing preliminary thinning pixels. 14. The method according to claim 8, characterized in that the exclusion of invalid pixels adjacent to the encoded block and invalid pixels adjacent to the reference block is carried out by performing the following additional operations:
- using all already restored pixels $$T_k^R(k=0...,N-\mathrm{one})$$

and all already restored pixels $$T_k^D$$

, k = 0, ..., N-1, form pairs $$\left(T_k^R,T_k^D\right)$$

, which are uniquely determined by the value of the pixel number k, then the resulting pairs are grouped into M disjoint groups G0 ... GM-1, such that $$\left(T_k^R,T_k^D\right)$$

, k = 0 ... N-1∈ {G ₀ ∩ ... G ₁ ∩ ... ∩ G _M-1 }, while the pairs of pixels belonging to the group Gi are also denoted as $$\{\left(T_{ki}^R,T_{ki}^D\right)\in Gi\}$$

;
- for each group Gi, a metric is calculated that is based on the consideration of pixel differences $$T_{ki}^R$$

and $$T_{ki}^D$$

except for the difference in average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each group Gi;
- for each group Gi, a metric is calculated based on differences in the average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each group Gi;
- calculate the general metric, denoted by MR_Norm, for groups G0 ... GM-1, which is based on the consideration of pixel differences in pixels $$T_{ki}^R$$

and $$T_{ki}^D$$

for each of the groups with the exception of differences in the average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each group Gi;
- calculate the general metric, denoted by M_Norm, for groups G0 ... GM-1, which is based on the difference in the average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each group Gi;
- comparing the values of the calculated metrics MR_Norm and M_Norm with predetermined threshold values, and if MR_Norm is greater than the first specified threshold value or M_Norm is less than the second specified threshold value, all pixels from a nearby neighborhood of the reference block and all pixels from a nearby neighborhood of the encoded block consider unreliable. 15. The method according to 14, characterized in that for each group Gi the metric MR_Norm (G _i ) is calculated, which is based on the consideration of pixel differences $$T_{ki}^R$$

and $$T_{ki}^D$$

except for the difference in average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each Gi group according to the formula:
$$MR\_Norm\left(G_i\right)={\displaystyle \sum _{\left(T_{ki}^R,T_{ki}^D\right)\in Gi}\left({|\; |\; |T_{ki}^R-T_{ki}^D-\frac{{\displaystyle \sum _{\left(T_{ki}^R,T_{ki}^D\right)\in Gi}{T}_{ki}^R}}{|\; |\; |Gi|\; |\; |}+\frac{{\displaystyle \sum _{\left(T_{ki}^R,T_{ki}^D\right)\in Gi}{T}_{ki}^D}}{|\; |\; |Gi|\; |\; |}|\; |\; |}^{P\mathrm{one}}\right)}$$

,
where the value of the power exponent P1 is determined experimentally;
- for each group Gi, the metric M_Norm (G _i ) is calculated based on the differences between the average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each Gi group according to the formula:
$$M\_Norm\left(G_i\right)={\displaystyle \sum _{\left(T_{ki}^R,T_{ki}^D\right)\in Gi}\left({|\; |\; |\frac{{\displaystyle \sum _{\left(T_{ki}^R,T_{ki}^D\right)\in Gi}{T}_{ki}^R}}{|\; |\; |Gi|\; |\; |}-\frac{{\displaystyle \sum _{\left(T_{ki}^R,T_{ki}^D\right)\in Gi}{T}_{ki}^D}}{|\; |\; |Gi|\; |\; |}|\; |\; |}^{P2}\right)}$$

,
where the value of the power exponent P2 is determined experimentally;
- calculate the general metric, denoted by MR_Norm, for groups G0 ... GM-1, which is based on the consideration of pixel differences in pixels $$T_{ki}^R$$

and $$T_{ki}^D$$

for each of the groups, with the exception of differences between the average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each Gi group according to the formula:
$$MR\_Norm={\displaystyle \sum _{i=0...M-\mathrm{one}}MR\_Norm(G_i)}$$

- calculate the general metric, denoted by M_Norm, for groups G0 ... GM-1, which is based on taking into account the difference between the average values of all pixels $$T_{ki}^R$$

and average values of all pixels $$T_{ki}^D$$

each Gi group according to the formula:
$$M\_Norm={\displaystyle \sum _{i=0...M-\mathrm{one}}M\_Norm(G_i)}$$

. 16. The method according to claim 1, characterized in that exclude unreliable pixels $$T_k^R$$

from a specific nearby neighborhood of the reference block and recovered pixels $$T_k^D$$

from a certain nearby neighborhood of the current encoded block by performing the following operations:
- for all pixels $$T_k^R$$

satisfying the condition $$T_k^R>L{R}_i$$

and $$T_k^R<L{R}_{i+\mathrm{one}}$$

pixels $$T_k^D$$

that match the pixels $$T_k^R$$

based on their serial number k = 0 ... N-1 and the spatial position in the reference and encoded frames, are combined into groups denoted by B (LR _i , LR _{i + 1} ):
$$B(L{R}_i,L{R}_{i+\mathrm{one}})=\{\underset{k=0...N-\mathrm{one}}{T_k^D}:T_k^R>L{R}_i\wedge T_k^R<L{R}_{i+\mathrm{one}}\}$$

,
the values LR _i , LR _{i + 1} determine the boundaries of the range defining the group B (LR _i , LR _{i + 1} ) and satisfy the condition LR _i >-1; LR _{i + 1} > R _i , the number of groups B (LR _i , LR _{i + 1} ); N _B sets the maximum possible value of the index i used in the numbering of quantities (LR _i , LR _{i + 1} ): -1 <LR ₀ <LR ₀ <LR ₀ ... <LR _NB ;
- for each group B (LR _i , LR _{i + 1} ) defined by the values (LR _i , LR _{i + 1} ), the following values are calculated:
$$C\_Plus\left(LRi,LRi+\mathrm{one}\right)={\displaystyle \sum _{T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)}\{\begin{array}{l}\mathrm{one,}{T}_k^R\ge T_k^D+Thr5\\ 0T_k^R<T_k^D+Thr5\end{array}}$$

$$C\_Minus\left(LRi,LRi+\mathrm{one}\right)={\displaystyle \sum _{T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)}\{\begin{array}{l}\mathrm{one,}{T}_k^R\le T_k^D-Thr5\\ 0T_k^R>T_k^D-Thr5\end{array}}$$

,
where Thr5 is the threshold value specified in advance or determined adaptively during the processing of the frame,
- for each pixel $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

groups B (LR _i , LR _{i + 1} ) verify the following three conditions:
- condition 1:
$$|\; |\; |C\_Plus\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)-C\_Minus\left(L{R}_{i,}L{R}_{i+\mathrm{one}}\right)|\; |\; |<Thr6$$

;
- condition 2:
$$C\_Plus\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)-C\_Minus\left(L{R}_{i,}L{R}_{i+\mathrm{one}}\right)>=Thr6AND{T}_k^R\ge T_k^D+Thr5$$

;
- condition 3:
$$C\_Plus\left(LRi,LRi+\mathrm{one}\right)-C\_Minus\left(LRi,LRi+\mathrm{one}\right)<=-Thr6AND{T}_k^R\le T_k^D-Thr5$$

in the event that one of the checked conditions 1 ... 3 is true for the next pixel in question $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

group B (LR _i , LR _{i + 1} ), the pixel in question $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

considered reliable. 17. The method according to clause 16, characterized in that the following operations are performed:
- for all pixels $$T_k^R$$

satisfying the condition $$T_k^R>L{R}_i$$

and $$T_k^R<L{R}_{i+\mathrm{one}}$$

pixels $$T_k^D$$

that match the pixels $$T_k^R$$

based on their serial number k = 0 ... N-1 and the spatial position in the reference and encoded frames, are combined into groups denoted by B (LRi, LR _{i + 1} ):
$$B(L{R}_i,L{R}_{i+\mathrm{one}})=\{\underset{k=0...N-\mathrm{one}}{T_k^D}:T_k^R>L{R}_i\wedge T_k^R<L{R}_{i+\mathrm{one}}\}$$

,
the values LR _i , LR _{i + 1} determine the boundaries of the range defining the group B (LR _i , LR _{i + 1} ) and satisfy the condition LR _i >-1; LR _{i + 1} > LR _i , the number of groups B (LR _i , LR _{i + 1} ) N _B sets the maximum possible value of the index i used for numbering the quantities (LR _i , LR _{i + 1} ): -1 <LR ₀ <LR ₀ <LR ₀ ... <LR _NB ;
- for all pixels $$T_k^D$$

belonging to group B (LR _i , LR _{i + 1} ), the following values are calculated:
○ mean value of Mean (B (LR _i , LR _{i + 1} )) for group B (LR _i , LR _{i + 1} ):
$$Mean\left(B\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)\right)=\frac{{\displaystyle \sum _{T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)}\left(T_k^D\right)}}{|\; |\; |B\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)|\; |\; |}$$

,
where | B (LR _i , LR _{i + 1} ) | denotes the number of pixels in group B (LR _i , LR _{i + 1} ),
○ average deviation Dev (B (LR _i , LR _{i + 1} )) of pixel values in group B (LR _i , LR _{i + 1} ):
$$Dev\left(B\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)\right)=\sqrt{\frac{{\displaystyle \sum _{T_k^D\in B\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)}{\left(T_k^D-Mean(B(L{R}_i,L{R}_{i+\mathrm{one}}))\right)}^2}}{|\; |\; |B\left(L{R}_i,L{R}_{i+\mathrm{one}}\right)|\; |\; |}}$$

;
if for the next pixel in question $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

of group B (LR _i , LR _{i + 1} ), the modulus of the difference between its value and the average value of group B (LR _i , LR _j ) is greater than the value Dev (B (LR _i , LR _j )) times greater than 0, which defines the jointly acceptable spread in group B (LR _i , LR _j ), the pixel in question $$T_k^D\in B\left(LRi,LRi+\mathrm{one}\right)$$

deemed unreliable. 18. The method according to 17, characterized in that the average deviation Dev (B (LR _i , LR _{i + 1} )) of the pixel values in group B (LR _i , LR _{i + 1} ) is calculated by the formula:
$$Dev\left(B\left(L{R}_i,L{R}_j\right)\right)=\frac{{\displaystyle \sum _{T_k^D\in B\left(L{R}_i,L{R}_j\right)}|\; |\; |T_k^D-Mean(B(L{R}_i,L{R}_j))|\; |\; |}}{|\; |\; |B\left(L{R}_i,L{R}_j\right)|\; |\; |}$$

. 19. The method according to claim 2, characterized in that:
- evaluate the possible change in each pixel value of the reference block based on a predetermined numerical criterion and at least two estimates estD _{i, j} and estR _{i, j} already calculated, determining it as significant or non-essential depending on the calculated value of the numerical criterion. 20. The method according to claim 19, characterized in that if the first estimate estD _{i, j} and the second estimate estR _{i, j} are such that the equality holds: EstR _{i, j} + Δ1 >> Qbits ₁ == EstD _{i, j} + Δ1 >> Qbits ₁ , where Qbits ₁ and Δ1 are non-negative integers, the degree of correction is estimated as insignificant. 21. The method according to claim 19, characterized in that if the first estimate estD _{i, j} and the second estimate estR _{i, j} are such that the equality holds: ( | EstD _{i, j} -EstR _{i, j} |  + Δ2)>> Qbits ₂ == 0, where Obits ₂ and Δ2 are non-negative integers, the degree of correction is estimated as insignificant. 22. The method according to claim 19, characterized in that if the first estimate estD _{i, j} and the second estimate estR _{i, j} are such that the following inequality holds: Δ3 × EstR _{i, j} <EstD _{i, j} <Δ4 × EstR _{i, j} , where Δ3, Δ4 are predefined non-negative numbers, the degree of correction is estimated as insignificant. 23. A method of adaptive compensation of local differences in brightness during inter-frame prediction of frames in the process of encoding and decoding a multi-angle video sequence, implemented by a computer system and including the following operations:
- get the pixel values of the current encoded block belonging to the encoded frame, and the pixel values of the reference block belonging to the reference frame;
- receive restored, that is, encoded and then decoded, pixel values from the vicinity of the current block of the encoded frame and pixel values from the neighborhood of the reference block of the reference frame;
characterized in that:
- exclude from consideration pixels that are estimated to be inaccurate for calculating brightness change parameters according to a predetermined at least one criterion and which belong to a neighborhood of the current block of the encoded frame and a neighborhood of the reference block of the reference frame;
- taking into account the initial values and using only reliable pixels, the brightness correction parameters are calculated;
- evaluate the change in pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters. 24. The method according to item 23, wherein the calculation of the brightness correction parameters includes the following operations:
- calculate at least two numerical estimates EstD and EstR based on the restored values $$T_k^D$$

pixels from the neighborhood of the current block being encoded and values $$T_k^R$$

pixels from the vicinity of the reference block, where N is the number of pixels adjacent to the current encoded and reference blocks, k = 0, ..., N-1;
- determining, on the basis of at least the first and second estimates EstD and EstR, a brightness correction parameter of all pixels of the reference block. 25. The method according to paragraph 24, wherein the procedure for calculating the first and second estimates includes the following operations:
- set the initial value of BaseR for the first evaluation and the initial value of BaseD for the second evaluation so that BaseR and BaseD are either both non-negative or both non-positive integers;
- calculate the first score $$estD=BaseD+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{T}_k^D}$$

where $$T_k^D$$

, k = 0, ..., N-1 - restored pixel values from a specific nearby neighborhood of the current encoded block, N - number of pixels in a predetermined nearby neighborhood of the encoded block;
- calculate the second score $$estR=BaseR+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{T}_k^R}$$

where and $$T_k^D$$

, k = 0, ..., N-1 - pixel values, from a certain nearby neighborhood of the reference block;
- if BaseR is equal to zero and the second EstR estimate is not equal to zero, or if BaseR is not equal to zero, the brightness correction parameter is determined as the ratio $$\alpha =\frac{EstD}{EstR}$$

;
- if BaseR is equal to zero and the second EstR estimate is zero, the parameter α is set to 1;
- perform the correction of the brightness of the reference block by multiplying the values of each pixel of the reference block R _{i, j} by a certain brightness correction parameter α if the parameter α is different from 1. 26. The method according to item 23, wherein:
- determine the vicinity of the current encoded block as nearby, so that it is directly adjacent to the encoded block, and the pixels included in it are already restored;
- determine the neighborhood of the reference block as nearby, so that it directly surrounds the reference block, the pixels included in it are already restored, the total number of pixels coincides with the total number of pixels in a given neighborhood of the current block being encoded, and between the pixels of the neighborhood of the reference block and the pixels of the neighborhood of the current the block being encoded a one-to-one correspondence is established:
$$estD=BaseD+{\displaystyle \sum _{k=0\_N-\mathrm{one}}{T}_k^{D}EstR=BaseR+}{\displaystyle \sum _{k=0\_N-\mathrm{one}}{T}_k^R}$$

$$T_k^R$$

, k = 0, ..., N-1 α α α. 27. The method according to p. 26, characterized in that the position of the vicinity of the current encoded block and the position of the neighborhood of the reference block is determined adaptively instead of nearby locations of the neighborhoods specified relative to the corresponding blocks in the encoded and reference frames. 28. The method according to p, characterized in that:
- evaluate possible changes in the pixel values of the reference block based on a predetermined numerical criterion and already calculated at least two estimates EstD, EstR, determining them as significant or insignificant depending on the calculated value of the numerical criterion. 29. The method according to p. 28, characterized in that if the first estimate of EstD and the second estimate of EstR are such that the equality holds: (EstR + Δ1) >> Qbits ₁ == (EstD + Δ1) >> Qbits ₁ , where Qbits ₁ and Δ1 are non-negative integers, the degree of correction is estimated as insignificant. 30. The method according to p. 28, characterized in that if the first estimate of EstD and the second estimate of EstR are such that the equality holds: (| EstD-EstR | + Δ2) >> Qbits ₂ == 0, where Qbits ₂ and Δ2 - non-negative integers, the degree of correction is estimated as insignificant. 31. The method according to p. 28, characterized in that if the first estimate of EstD and the second estimate of EstR are such that the following inequality holds: Δ3 × EstR <EstD <Δ4 × EstR, where Δ3, Δ4 are non-negative integers, the degree of correction is estimated as inconsequential. 32. The method according to item 23, wherein the brightness correction parameter α, for simplicity, is a non-negative integer such that the correction of each pixel value is calculated by at least multiplying each pixel value by a non-negative integer representing the integer brightness correction parameter for all pixels of the reference block and a bit shift to the right by another predetermined non-negative integer. 33. The method according to item 23, wherein:
- rate pixels $$T_k^R$$

, $$T_k^D$$

as unreliable if for the value k = 0 ... N-1 the inequality $$|\; |\; |T_k^R-T_k^D|\; |\; |\le Thr\mathrm{one}$$

where Thr1 is a predetermined numerical threshold;
- exclude pixels from further calculations $$T_k^R$$

, $$T_k^D$$

which are rated as unreliable. 34. The method according to item 23, wherein when obtaining the restored pixel values from the vicinity of the current and reference blocks, thinning is performed according to a predetermined pattern. 35. The method of encoding multi-angle video sequences, based on adaptive compensation of local differences in brightness during inter-frame prediction of frames, in which the encoded frame is a set of non-overlapping areas of pixels acting as coding units, the method being implemented by a computer system and includes the following operations:
- choose the current encoded block belonging to the current coding unit;
- determine at least one reference block, which is used to generate a prediction block for the current block being encoded and whose pixels are already fully encoded and decoded;
- determine the brightness correction parameters of the reference block;
- perform brightness correction by adjusting the values of all pixels of the reference block based on certain brightness correction parameters;
- form a prediction block for the current encoded block using the adjusted reference block:
- encode the current block using the generated prediction block;
characterized in that the steps of determining brightness correction parameters and performing brightness correction include the following:
- receive restored, that is, encoded and then decoded, pixel values from the vicinity of the current block of the encoded frame and pixel values from the neighborhood of the reference block of the reference frame;
- exclude from consideration pixels that are estimated to be inaccurate for calculating brightness correction parameters according to a predetermined at least one criterion and which belong to a neighborhood of the current block of the encoded frame and a neighborhood of the reference block of the reference frame;
- calculate the brightness correction parameters, taking into account the initial values and using only reliable pixels;
- evaluate the change in pixel values of the reference block; if the change is assessed as significant, correct the pixel values of the reference block using the found correction parameters. 36. The method according to p. 35, characterized in that when selecting the current encoded block belonging to the current coding unit, determine the size of the current encoded block, as well as the relative location and size of the neighborhood of the encoded and reference blocks, the pixels of which are used to calculate correction parameters based on encoding parameters of the current encoding unit. 37. The method according to clause 36, wherein the size of the current encoded block, as well as the relative location and size of the neighborhood of the encoded and reference blocks, the pixels of which are used to calculate the correction parameters, is determined based on the type of decorrelation transformation used. 38. The method according to clause 36, wherein the size of the current encoded block is determined based on a predetermined division of the coding unit into subunits having separate motion vectors. 39. The method according to clause 36, wherein the size of the current encoded block must be less than the size of the sub-block, which belongs to the current coding unit and has a separate motion vector. 40. The method according to clause 35, wherein when selecting the current encoded block belonging to the current coding unit, the size of the current encoded block is explicitly determined and encoded in the stream. 41. The method according to p. 35, characterized in that when determining the reference block, which is used to generate the prediction block for the current encoded block, from all the reference frames used to search for the reference block, only one reference frame is selected according to predetermined rules; they do not encode information about the selected reference frame. 42. The method according to paragraph 41, wherein when selecting only one reference frame, a synthesized frame is selected for the current view with the same time stamp as the encoded frame. 43. The method according to paragraph 41, wherein when selecting only one reference frame, a reference frame belonging to one of the neighboring views and having the same time stamp as the encoded frame is selected. 44. The method according to clause 35, wherein when determining the reference block, which is used to generate the prediction block for the current encoded block, determine the motion vector based on the similarity function, which further takes into account differences in the average brightness levels of the current encoded block and the reference candidate block. 45. The method according to item 44, wherein in determining the motion vector, calculate the similarity function as follows:
- the coded block and the reference block are divided into subblocks, while the sizes of the subblocks do not exceed the sizes of the coded and reference blocks;
- the average brightness level for each sub-block of the encoded and reference block is calculated;
- adjust each sub-block of the encoded and reference block; for this, the average brightness level corresponding to it is subtracted from each pixel of the subblock;
- calculate the sum of the absolute differences of the adjusted reference and the corrected encoded block;
- calculate the sum of the absolute differences for the average levels of all sub-blocks of the reference and encoded block;
- calculate the similarity index as the sum of the absolute differences of the average levels of the subunits, multiplied by the first non-negative number, added to the sum of the absolute differences of the adjusted blocks, multiplied by the second non-negative number. 46. The method according to clause 35, wherein the reference block is determined using the displacement vector, while limiting the maximum possible distance between the displacement vector and the prediction of the displacement vector so that the difference of the horizontal components of the displacement vector and the forecast of the displacement vector, called horizontal projection, and the difference between the vertical components of the displacement vector and the forecast of the vector, called the vertical projection of the displacement, have various limitations. 47. The method according to item 46, wherein in determining the restrictions for horizontal projection and vertical projection, the restriction of vertical projection does not exceed the restrictions for horizontal projection. 48. The method according to clause 35, wherein when encoding the current block using the generated prediction block, an additional syntax element is encoded for the encoding unit to which the encoded block belongs; the value of the syntax element determines whether changes in the pixel values of the reference blocks of the coding unit are significant or not. 49. The method according to claim 35, characterized in that when encoding the current block using the generated prediction block, one additional syntax element is encoded for each coding unit sub-block, which may have a unique reference frame index in a given coding unit; the value of the syntax element determines whether changes in the pixel values of the reference block are significant or not. 50. The method according to p. 35, characterized in that when encoding the current block using the generated prediction block for the encoding unit to which the encoded block belongs, an additional syntax element is encoded, the value of which can take at least two values; when one value of the encoded additional syntax element is used when determining the reference block, motion vector prediction is used, adapted to the method of correcting pixel values of the reference block and calculating brightness correction parameters and correcting pixel values of the reference block; at a different value of the encoded additional syntax element, the standard prediction of the motion vector is used to determine the reference block and the brightness correction parameters and the correction of the pixel values of the reference block are not calculated. 51. The method according to p. 50, characterized in that the prediction of the motion vector, adapted to the method of correction of pixel values of the reference block, is performed only in the reference frame of the neighboring view. 52. The method according to 51, wherein the motion vector prediction is performed using previously found motion vectors pointing to a reference frame of a neighboring view and using already encoded depth maps. 53. The method according to clause 35, wherein when encoding the current block, the calculated brightness correction parameters are not encoded. 54. The method according to clause 35, wherein additional brightness correction parameters are calculated based on the pixel values of the reference block and the pixels of the encoded block, which are used to correct the pixel values of the reference block, and when encoding the current block, additional brightness correction parameters are encoded using the parameters correction of brightness calculated on the basis of the restored pixel values from the vicinity of the encoded block and pixel values from the vicinity of the reference block. 55. The method of decoding multi-angle video sequences, based on adaptive compensation of local differences in brightness during inter-frame prediction of frames, in which the decoded frame is a set of non-overlapping areas of pixels acting as coding units, the method being implemented by a computer system and includes the following operations:
- choose the current decoded block belonging to the current coding unit;
- determine the reference block for the current decoded block;
- determine the correction parameters of the change to correct the brightness of the found reference block;
- perform brightness correction by adjusting the values of all pixels of the reference block based on certain brightness correction parameters;
- form a prediction block for the current decoded block using the adjusted reference block;
- decode the current block using the generated prediction block;
characterized in that the steps of determining brightness correction parameters and performing brightness correction include the following:
- receive restored, that is, encoded and then decoded, pixel values from the vicinity of the current block of the decoded frame and pixel values from the neighborhood of the reference block of the reference frame;
- exclude from consideration the pixels belonging to the vicinity of the current decoded block and the neighborhood of the reference block of the reference frame, which are estimated as invalid for calculating the parameters of the brightness change according to a predetermined at least one criterion;
- taking into account the initial values, the values of the brightness correction parameters are calculated based on the found numerical ratios;
- carry out the correction of the pixel values of the reference block using the found correction parameters in the event that changes in the pixel values of the reference block are evaluated as significant. 56. The method according to item 55, wherein when selecting the current decoded block belonging to the current coding unit, determine the size of the current decoded block, as well as the relative location and size of the neighborhood of the decoded and reference blocks, the pixels of which are used to calculate correction parameters based on encoding parameters of the current encoding unit. 57. The method according to p. 56, characterized in that the size of the current decoded block, as well as the relative location and size of the neighborhood of the decoded and reference blocks, the pixels of which are used to calculate the correction parameters, are determined based on the type of decorrelation transformation used. 58. The method according to p, characterized in that the size of the current decoded block is determined based on a predetermined division of the coding unit into subunits having separate motion vectors. 59. The method according to p, characterized in that the size of the current decoded block should be less than the size of the sub-block, which belongs to the current coding unit and has a separate motion vector. 60. The method according to item 55, wherein when selecting the current decoded block belonging to the current coding unit, determine the size of the current decoded block based on the decoded information from the input bit stream. 61. The method according to item 55, wherein in determining the reference block, which is used to generate the prediction block for the current decoded block, from all the reference frames, only one reference frame is selected according to predetermined rules; however, information about the selected reference frame is not contained in the input bitstream. 62. The method according to p, characterized in that when selecting a reference frame, a synthesized frame is selected for the current view with the same time stamp as the decoded frame. 63. The method according to p, characterized in that when selecting a reference frame select a reference frame belonging to one of the neighboring angles, and having the same time stamp as the decoded frame. 64. The method according to claim 55, characterized in that when decoding the current block using the generated prediction block, an additional syntax element is decoded for the coding unit to which the decoded block belongs; the value of the syntax element determines whether significant changes in the pixel values of the reference blocks of the coding unit or not. 65. The method according to claim 55, characterized in that when decoding the current block using the generated prediction block, one additional syntax element is decoded for each coding unit sub-block, which may have a unique reference frame index in a given coding unit; the value of the syntax element determines whether changes in the pixel values of the reference block are significant or not. 66. The method according to claim 55, characterized in that when decoding the current block using the generated prediction block, an additional syntax element is decoded for the coding unit to which the decoded block belongs, the value of which can take at least two values, with one of which, in determining the reference block, motion vector prediction is used, adapted to the method of correcting pixel values of the reference block, and the brightness correction parameters are calculated and pixel values are corrected leu support block; for a different value of the decoded additional syntax element, the standard prediction of the motion vector is used to determine the reference block and the calculation of the brightness correction parameters and the correction of the pixel values of the reference block are not performed. 67. The method according to p, characterized in that the prediction of the motion vector, adapted to the method of correction of pixel values of the reference block, is performed only in the reference frame of the neighboring view. 68. The method according to p, characterized in that the prediction of the motion vector is performed using previously found motion vectors pointing to the reference frame of a neighboring view and using already encoded depth maps. 69. The method according to item 55, wherein when decoding the current block, information about the brightness correction parameters is not contained in the input bit stream. 70. The method according to claim 55, characterized in that, when decoding the current block, additional brightness correction parameters located in the input bitstream are decoded using the brightness correction parameters calculated based on the restored pixel values from a neighborhood of the encoded block of pixel values from a reference neighborhood block. 71. A method for decoding multi-angle video sequences based on adaptive compensation of local differences in brightness during inter-frame prediction of frames, in which the decoded frame is a set of non-overlapping areas of pixels called coding units, the method being implemented by a computer system and includes the following operations:
- choose the current decoded block belonging to the current coding unit;
- determine the reference block for the current decoded block;
- determine the correction parameters of the change to correct the brightness of the found reference block;
- perform brightness correction by adjusting the values of all pixels of the reference block based on certain brightness correction parameters;
- form a prediction block for the current decoded block using the adjusted reference block;
- decode the current block using the generated prediction block;
characterized in that the steps of determining brightness correction parameters and performing brightness correction include the following:
- receive restored, that is, encoded and then decoded, pixel values from the vicinity of the current block of the decoded frame and pixel values from the neighborhood of the reference block of the reference frame;
- exclude from consideration the pixels belonging to the vicinity of the current decoded block and the neighborhood of the reference block of the reference frame, which are estimated as invalid for calculating the parameters of the brightness change according to a predetermined at least one criterion;
- determine the numerical relationship between the pixels of the reference block, valid pixels from the vicinity of the current decoded block and reliable pixels from the neighborhood of the reference block;
- taking into account the initial values, the values of the brightness correction parameters are calculated based on the found numerical ratios;
- carry out the correction of the pixel values of the reference block using the found correction parameters in the event that changes in the pixel values of the reference block are evaluated as significant.

Images