RU2557449C1 - Video encoding and decoding method based on three-dimensional discrete cosine transform - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: adaptation is performed by rearranging fragments of discrete cosine transform (DCT) coefficients obtained after two-dimensional DCT on the time axis and subsequent one-dimensional DCT such that the total number of non-zero transform coefficients after three-dimensional DCT is less than the number of non-zero DCT coefficients obtained after three-dimensional DCT without rearranging two-dimensional DCT fragments. In the disclosed method, after forming a domain measuring n×n×n pixels, DCT coefficients are calculated on spatial coordinates x and y for each fragment of the domain. The fragments are then rearranged in the form of a rearrangement vector and a time DCT operation is performed. The DCT coefficients are sampled, encoded and transmitted over a communication channel with the rearrangement vector. At reception, said procedures are performed in reverse order and the original video stream is restored.

EFFECT: high degree of compression of video data with a given image reconstruction error at reception owing to adaptation to variation of static properties of images.

3 cl, 9 dwg

Description

The invention relates to the field of coding, and in particular to a method for compressing moving images in order to reduce the amount of data necessary for their storage or transmitted over a communication channel for subsequent restoration of images on reception.

A known coding method based on three-dimensional discrete cosine transform (DCT) [Zaharia R., Aggoun A., McCormick M. Adaptive 3D-DCT compression algorithm for continuous parallax 3D integral imaging. Journal of Signal processing: Image Communication. 17, pp. 231-242, 2002].

The disadvantage of this method is the lack of adaptation in the encoding process to the degree of mobility of the frames of the moving image, which leads to an increase in the amount of data at the output of the encoder for a given encoding error.

A known method of video compression, which uses three-dimensional DCT (DCT-3) [Bozinovic N., Konrad J. Scan or derandquantization for 3D-DCT coding in Proc. of SPIEV is. Comm. Andlm. Proc. Vol. 5150. pp. 1204-1215, 2003].

In this method, processing of a video sequence of frames based on three-dimensional DCT is used. In this case, the conversion operation begins with the spatial coordinates x and y, and the obtained spectral coefficients of the two-dimensional DCT (DCT-2) are subjected to the one-dimensional DCT (DCT-1) along the time coordinate t to reduce time redundancy.

The main disadvantage of the analogue method is the inability to adapt coding operations to the statistics of the original images, which makes it impossible to achieve high compression ratios.

The closest in technical essence to the claimed method is the "Method of encoding and decoding video information based on three-dimensional discrete cosine transform" patent RU No. 2375838, publ. 12/10/2009, bull. Number 34. The prototype method consists in the following actions: packets of n frames are divided into transmission sequences of television frames, from which domains of size n × n × n pixels are formed, then at the first stage of coding, each domain of size n × n × n pixels performs DCT in time , determine the presence of motion in each fragment of size n × n pixels based on the presence of non-zero spectral coefficients, except for the first fragment of the domain, in the case of movement in each fragment of the domain to eliminate spatial ti is calculated DCT coefficients (MPC) in two spatial coordinates x and y obtained coefficients are quantized, encoded set of quantized coefficients to eliminate spatial redundancy, the coded coefficients are transmitted to the communication channel. In the absence of movement, the CDQP is calculated by the spatial coordinates x and y only for the first fragment of the domain and the operations of quantization, coding, and transmission of coefficients over the communication channel are performed. At the subsequent stages of coding, when the next packets arrive, if there is movement, then the coding process is repeated, if there is no movement in specific domains, then a signal about the previous fragment is used for them when decoding. At the reception, the compressed video stream is decoded and then dequantized. In the case of movement in the domains, the dequantized coefficients are subjected to reverse DCT-3 (ODKP-3) (sequential execution of the inverse two-dimensional DCT (ODKP-2D) and reverse one-dimensional DCT (ODKP-1D) and, as a result, the original video stream is restored. specific domains using the spectral coefficients transferred in previous domains (stored in the fragment buffer without movement) restore fragments of these domains when performing only the inverse discrete cosine transform (DCCT) in time and in p result of restoring the original video stream.

The disadvantage of the prototype is that when encoding, accounting for the degree of mobility of the source images occurs by classifying them into two groups: with no movement and with the presence of movement. This makes the encoding method less efficient from the standpoint of achieved compression for a given reception recovery error due to insufficient adaptation to the degree of mobility of the encoded fragments of the original image.

The aim of the invention is to develop a method of encoding and decoding video information based on DCT-3, which provides an increase in the degree of compression of video data for a given error of image recovery at the reception due to adaptation to changes in the statistical properties of input images. Adaptation to change the statistical properties of the input images is carried out by rearranging the order of the CDCF fragments obtained after performing DCT-2 along the time axis, subsequent execution of DCT-1 in such a way that the total number of non-zero transformation coefficients after performing DCT-3 becomes less than the number nonzero conversion coefficients obtained after performing DCT-3 without performing permutation of fragments of DCT-2.

In the claimed method of encoding and decoding video information based on DCT-3, the goal is achieved in that in the known method of encoding and decoding video information based on DCT-3, which consists in compressing a sequence of television frames, for which this sequence is divided into packets by n frames from which domains of size n × n × n pixels are formed, DCT-3 is performed over each domain of size n × n × n to eliminate temporal and spatial redundancy, the obtained CDCs are quantized, encoded To eliminate statistical redundancy, a compressed video stream is received from a communication channel, a compressed video stream is subjected to decoding, dequantization, ODKP-3, as a result, the original video stream is restored from the compressed signal. In this case, after the formation of a domain of size n × n × n pixels, the CDQP is calculated by the spatial coordinates x and y for each fragment of the domain. Then perform the rearrangement of the fragments obtained KDKP spatial coordinates x and y. Remember the permutation in the form of a permutation vector. After that, the DCT operation is performed in time, the CDQP is quantized and encoded. Next, the obtained quantized QDKP and permutation vector are transmitted to the communication channel, received from the communication channel, decoded QDKP and vector of permutation, and QDKW are de-coupled. Above the dequantized CDQDs, the DCTCH operation is performed in time. Perform the reverse permutation of the CDCF fragments by the spatial coordinates x and y. A domain of size n × n × n pixels is restored by calculating the DCCT coefficients from the spatial coordinates x and y, and as a result, the original video stream is restored.

To rearrange the fragments of the obtained QDCP in spatial coordinates x and y, the DCT operation in time is preliminarily performed on the obtained QCD in spatial coordinates x and y. Next, the obtained co-QCDCs are quantized, the number NZ of nonzero quantized QCDCs is determined, and stored. Then, in succession, each fragment of the calculated QCDCs at the spatial coordinates x and y is alternately moved to the location of the remaining fragments of the calculated QCDCs at the x and y coordinates, and at the place of the moved fragment of the calculated QCDCs at the x and y coordinates, a fragment of the calculated QCDCs at the x and y coordinates is placed instead which located the displaced fragment of the calculated CDQP at the x and y coordinates. Then, for the obtained QDQDs, the DCT operation is performed in time, the number NZt of nonzero quantized QDQDs is quantized, and determined. If NZt turns out to be less than the previously memorized NZ, then reassignment NZ = NZt is performed and the order of the fragments of the calculated QDQPs in spatial coordinates x and y is stored in the form of a permutation vector. Otherwise, restore the previous location of the fragments of the calculated KDKP by spatial coordinates x and y.

To memorize the order of the location of the CDCF fragments by the spatial coordinates x and y, a permutation vector of 1 × n elements is formed by assigning each of its elements P _i , where i = 1, 2, ..., n, the location number of the corresponding fragment of the calculated CDCC by the spatial coordinates x and y.

Thanks to the new set of essential features in the claimed method, the indicated technical result is achieved by changing the order of the location along the time axis of the fragments of the calculated QDQS at the x and y coordinates so that the total number of nonzero quantized QDQDs after the three-dimensional transformation was minimal.

The claimed method is illustrated by drawings, which show:

figure 1 is a structural diagram of the claimed method of encoding and decoding video information based on DCT-3;

figure 2 - the essence of the claimed method based on DCT-3;

figure 3 - the formation of the source domain in the form of a three-dimensional array of size n × n × n pixels;

figure 4 - calculation of the CDQP by the spatial coordinates x and y;

figure 5 is an example of a matrix DCT-2 with a size of 8 × 8 elements;

in Fig.6 is an example of DCT-1 over the calculated CDQA in spatial coordinates x and y without rearrangement and with rearrangement of fragments;

Fig. 7 is an example of quantized QDQDs, after performing a one-dimensional DCT without rearranging fragments of the calculated QDLC in spatial coordinates x and y along the time axis;

Fig. 8 is an example of quantized CDQPs after DCT-1 with permutation of fragments of the calculated CDQPs along spatial coordinates x and y along the time axis;

Fig.9 is an example of the dependences of the number of nonzero quantized QDQDs on the fragment number.

The ability to implement the claimed method of encoding and decoding video information based on DCT-3 is explained as follows.

Frames of a moving image are characterized by both intraframe or spatial redundancy, and interframe or temporal redundancy. As a rule, in the well-known compression standards H.263, H.264, some decorrelation transformation, for example, DCT-2, is used to eliminate intraframe redundancy. To eliminate inter-frame redundancy, inter-frame prediction based on the transmission of motion vectors is used. In encoding methods based on DCT-3, elimination of intraframe and interframe redundancy is carried out by decorrelation of pixels of the original image both in spatial coordinates x and y, and along the time axis. As a result of decorrelation, most of the DCT-3 coefficients turn out to be zero or close to zero, which ensures a decrease in the required number of bits required for encoding DCT-3 coefficients. However, as practical studies show, the decorrelation transformation based on cosine functions is optimal only for a given class of images. This class of images is limited to low-frequency (with a small number of small details) images and, in the case of mobile video, images with a slow change in scenes when moving from frame to frame. In practice, the non-optimality of the transforms used, especially in the time domain, is manifested in the preservation of a large number of nonzero quantized transform coefficients, which, in turn, reduces the achieved compression ratio. The elimination of this drawback is possible on the basis of the implementation of the adaptation procedure in the coding process in two ways. The first is to change the decorrelation transform used, which would take into account the dynamics of changes in frame differences. The second consists in changing the properties of the input data, over which a fixed decorrelation transformation is performed. In this case, the conversion of the input data is performed in such a way as to bring them to a form optimal for the decorrelation conversion used.

Using the first method is problematic. This is due not only to the need to solve the complex problem of calculating the optimal conversion, but also to the necessity of transmitting large amounts of data describing the obtained transformation to a decoding device via a communication channel with limited bandwidth. Therefore, in the inventive method, an approach is proposed based on changes in the properties of the input data while maintaining unchanged used DCT as decorrelating. It is proposed to change the properties of the input data based on a change in the sequence of fragments of the coefficients calculated on the basis of DCT-2 along the time axis.

Clearly, the main idea of the proposed method is shown in figure 2. In the upper left corner is an example of a domain with a size of 8 × 8 × 8 pixels. In this case, the 1st, 2nd, 4th and 8th fragments of the domain consist of the same pixels equal to 255 (shown in white), and the 3rd, 5th, 6th and 7th fragments of the domain consist of identical pixels equal to 127 (shown in gray). After performing the DCT-2 operation on each fragment of the domain, 8 CDQP fragments were obtained at the x and y coordinates (upper right corner of FIG. 2). All the calculated QDECs at the x and y coordinates of each fragment are equal to zero, except for the coefficients with the coordinates x = 1 and y = 1, equal to 2040, 2040, 1016, 2040, 1016, 1016, 1016 2040 for 1, 2, 3, 4, 5 , 6, 7, and 8 fragments, respectively. After performing DKP-1 on the calculated CDKPs at the x and y coordinates without rearranging the fragments, 8 nonzero quantized CDKPs were obtained: 4321, 525, 669, -384, 724, -76, -277, -786. On the other hand, if we rearrange the fragments of the calculated QDQPs along the x and y coordinates, interchanging the 2nd and 5th fragments, we will get 2 nonzero quantized QDQDs: 4322, 0, 0, 0, 1448, 0, 0, 0 Thus, the permutation of fragments of the calculated QDQDs along the x and y coordinates leads to a 4-fold decrease in the number of nonzero QDQDs in this example, which, in turn, leads to an increase in the compression coefficient by approximately the same factor.

The implementation of this idea can be explained in the diagram shown in figure 1. The encoder receives domains in the form of three-dimensional arrays of pixels of size n × n × n. The formation of n × n × n pixel domains from a packet consisting of n frames of a moving image is shown in FIG. 3. Then, above each of the n fragments of the domain, the CDQP is calculated at the x and y coordinates, i.e. perform the operation DCT-2, as shown in figure 4. This operation is performed in block 11 (DCT-2D figure 1). Implementation of DCT-2 is carried out, for example, as described in the book: Ahmed N., Rao K. Orthogonal transformations in the processing of digital signals / Ed. I.B. Fomenko; Per. from English - M .: Svyaz, 1980. The matrix record DKP-2 of the i-th fragment of the source domain, represented by the matrix [A] _i , has the form:

m=1, 2,…, n; k=2,3,…,n. На фиг.5 показан пример матрицы ДКП-2 размером 8×8 элементов.where [S] _i - calculated KDKP on spatial coordinates x and y of the i-th fragment; [Г] and [Г] ^T - direct and transposed (inverse) DKP-2 matrices defined by an array of vectors $$\left\{\frac{\mathrm{one}}{\sqrt{n}},\sqrt{\frac{2}{n}\cos \frac{\left(2m-\mathrm{one}\right)\left(k-\mathrm{one}\right)\pi }{2n}}\right\},$$

m = 1, 2, ..., n; k = 2,3, ..., n. Figure 5 shows an example of a matrix DKP-2 with a size of 8 × 8 elements.

Further, fragments of the calculated CDQP along the x and y coordinates go to the permutation block. The permutation of the fragments is performed so that the number of nonzero QCDs obtained after performing DCT-1 along the time axis is minimal. This operation is performed in block 12 (permutation of figure 1). The permutation of the fragments is performed on the basis of the permutation vector, which is calculated in side 16 (Permutation control block of FIG. 1) depending on the parameters NZ and NZt obtained from the output of the quantization block. After that, the DCT operation is performed in time in block 13 (DCT-1D of FIG. 1). The obtained CDQPs are quantized in block 14 (quantization of FIG. 1) and encoded in block 15 (encoding of FIG. 1). Next, the obtained quantized QWCD and the permutation vector are transmitted to the communication channel (block communication channel of FIG. 1), received from the communication channel. Decode CDKP and permutation vector. This operation is performed in block 21 (decoding of FIG. 1). Then, the CDC is de-quantized in block 22 (de-quantization of FIG. 1). Above the dequantized CDC, the DCCT operation is performed in time in block 23 (DCC-1D of FIG. 1). After this, the reverse permutation of the CDCF fragments is performed by the spatial coordinates x and y in block 24 (permutation of FIG. 1). Next, a domain of size n × n × n pixels is restored by calculating the DCCT coefficients from the spatial coordinates x and y in block 25 (DCCT-2D of FIG. 1) and, as a result, the original video stream is restored.

For clarity, Fig. 6 on the left side shows the result of performing DCT-1 on the calculated CDQDs along spatial coordinates x and y without rearrangement, i.e. 1 fragment of the CDAC is in the first place, the second - in the second, etc. The permutation vector in this case has the form P = [1 2 3 4 5 6 7 8]. The number of nonzero quantized QDQDs in this example was 107. The right-hand side of Fig. 6 shows the result of performing DCT-1 on the calculated QDQDs according to the spatial coordinates x and y after their rearrangement. The found permutation vector in this example has the form P = [7 5 4 3 1 6 2 8], i.e. 7 fragment KDKP is located in the first place, 5 - in the second, etc. in accordance with the permutation vector. The number of nonzero quantized QDQDs in this case was 88, which is less than in the first case. Examples of quantized QDQDs, after performing DKP-1 without rearranging fragments of the calculated QDLCs in spatial coordinates x and y along the time axis and permutation are shown in Figs. 7 and 8, respectively.

To assess the effectiveness of the proposed method of encoding and decoding based on DCT-3, simulation was carried out on a PC. As a performance indicator, we used the coefficient of decreasing the number of nonzero quantized QDQDs when rearranging fragments of the calculated QDQPs in spatial coordinates x and y relative to the number of nonzero quantized QDQPs obtained without permutation.

A set of test images of 576 × 720 pixels in YUV 4: 4: 4 format and a frame rate of 25 frames / s were used as initial moving images. The size of the source domain was 8 × 8 × 16 pixels.

Figure 9 shows the characteristic dependences of the number of nonzero quantized QDQDs on the fragment number:

a) without rearrangement of fragments;

b) with rearrangement of fragments;

c) the magnitude of the difference between the number of KDKP obtained with permutation and without permutation.

We define the efficiency coefficient as $$K_{\mathrm{uh}f}=\left(\frac{N_{bP}-N_P}{N_{bP}}\right)\cdot \mathrm{one\; hundred}\%,$$

N _bp - the total number of nonzero quantized QCDP obtained without rearrangement of fragments; N _p - the total number of nonzero quantized QCDC obtained with the permutation of fragments.

The results of simulation of the developed method showed that the gain was 15 ÷ 20% compared with the prototype. Studies have shown that a decrease in the number of nonzero quantized QDQDs by 15–20% leads to the same increase in the compression ratio while maintaining the previous quality of the reconstructed images, which confirms the achievement of the purpose of the invention.

Claims (3)

1. A method of encoding and decoding video information based on three-dimensional discrete cosine transform (DCT), which consists in compressing a sequence of television frames, for which this sequence is divided into packets of n frames, from which domains of n × n × n pixels are formed, over each domain of size n × n × n pixels, three-dimensional DCT is performed to eliminate temporal and spatial redundancy, the obtained DCT coefficients are quantized, encoded to eliminate statistical redundancy, and transmitted the compressed video stream is received from the communication channel, the compressed video stream is decoded, dequantized, the inverse of the three-dimensional DCT, as a result, the original video stream is restored from the compressed signal, characterized in that, after the formation of a domain of n × n × n pixels, DCT coefficients are calculated by spatial the x and y coordinates for each domain fragment, then they rearrange the fragments of the obtained DCT coefficients by the spatial coordinates x and y, remember the permutation in the form of a permutation vector , perform the DCT operation in time, DCT coefficients are quantized, encode the obtained quantized DCT coefficients and a permutation vector and transmit to the communication channel, receive from the communication channel, decode DCT coefficients and a permutation vector, deduct DCT coefficients, perform reverse DCT operation on dequantized DCT coefficients time, perform reverse permutation of fragments of DCT coefficients by spatial coordinates x and y, restore a domain of size n × n × n pixels by calculating the coefficients of the inverse DCT on spatial coordinates x and y and as a result restore the original video stream. 2. The method according to claim 1, characterized in that to rearrange the fragments of the obtained DCT coefficients in the spatial coordinates x and y, the DCT operation in time is performed on the obtained DCT coefficients in the spatial coordinates x and y, the obtained DCT coefficients are quantized, the number NZ is determined quantized DCT coefficients and remember it, then each fragment of the calculated DCT coefficients in spatial coordinates x and y is successively moved to the location of the remaining fragments of the calculated DCT coefficients at the x and y coordinates, and at the place of the moved fragment of the calculated DCT coefficients at the x and y coordinates, there is a fragment of the calculated DCT coefficients at the x and y coordinates, instead of which there is a displaced fragment of the calculated DCT coefficients at the x and y coordinates, for the obtained after moving the DCT coefficients, perform the DCT operation in time, quantize and determine the number NZt of nonzero quantized DCT coefficients and, if NZt is less than the previously stored NZ, then perform assignment NZ = NZt and remember the order of fragments of the calculated DCT coefficients in the spatial coordinates x and y in the form of a permutation vector; otherwise, the previous arrangement of fragments of the calculated DCT coefficients in the spatial coordinates of x and y is restored. 3. The method according to claim 1, characterized in that for storing the order of the fragments of the calculated DCT coefficients by the spatial coordinates x and y form a permutation vector of 1 × n elements by assigning each of its elements P _i , where i = 1, 2, ... , n are the numbers of the location of the corresponding fragment of the calculated DCT coefficients with respect to the spatial coordinates x and y.

Images