RU2616176C1 - Method for encoding-decoding of digital video images - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: method for encoding-decoding of digital video images. According to the metod, in the process of encoding of the wavelet transform to the low frequency component for smoothing the initial function an additional high frequency component is added rowwise and used for encoding, but suppressed on the decoding side by using a lowpass filter. And encoding is implemented using a composite function with two goals of increasing data compression ratio and maintaining the quality of the decoded image, and the decoder filter characteristics are considered as a restriction at the encoding stage.

EFFECT: increased ratio of video image compression with a slight decrease of the decoded image quality with respect to the images having a high frequency nature of the signal spectrum.

8 dwg, 3 tbl

Description

The technical solution relates to the field of digital signal processing, in particular to methods of encoding-decoding digital video images, and can be applied in storage and processing of visual information.

There is a similar method of encoding-decoding digital video images based on the MPEG-4 encoding standard described in the book by Jan Richardson, “H.264 Video Encoding and MPEG-4 - New Generation Standards” Moscow: Technosphere, 2005. - 368 p., P. 197 and consisting of the following: when encoding each successive image, the encoder polls the color image sensor, then the resulting image is represented by three source color matrices corresponding to the red, green and blue color components of the image, after which the original color matrices are stored in the encoder memory, then to each source wavelet transform is applied row by row to the color matrix, during which the low-frequency and high-frequency components of the matrix are obtained, and then to the low-frequency the component of the current matrix is sequentially applied quantization and coding with spatial prediction, then quantization, scanning and coding according to the zero-tree method are applied to the high-frequency component of the current matrix, and then entropy arithmetic coding is applied to the preliminary codes of both frequency components of the current matrix, after encoding all the source matrices the set of received codes is written to the storage device by the decoder in the process of decoding the sn At the beginning, a set of image codes is extracted from the storage device, then for each decoded color matrix, the entropy arithmetic decoding is first applied to the obtained codes, as a result of which separate codes of the low-frequency and high-frequency components of the matrix are obtained, after which spatial prediction decoding is sequentially applied to the low-frequency component of the current matrix, and dequantization, then decoding by the method of zero d is applied to the high-frequency component of the current matrix revev, dequantization and reverse scan, and then decoded by the low-frequency and high-frequency components by the inverse wavelet transform is reduced starting current matrix image, and after decoding all color matrixes restore the original image.

The disadvantage of this analogue is the low value of the compression ratio of video images having a high-frequency character of the signal spectrum.

As a prototype, a method of encoding-decoding digital video images based on the MPEG-4 encoding standard described in the book by Jan Richardson, “H.264 Video Encoding and MPEG-4 — New Generation Standards,” Moscow: Technosphere, 2005. — 368 p., P. 197 and consisting of the following: when encoding each successive image, the encoder polls the color image sensor, then the resulting image is represented by three source color matrices corresponding to the red, green and blue color components of the image, after which the original color matrices are stored in the encoder memory, then to each source wavelet transform is applied row by row to the color matrix, during which the low-frequency and high-frequency components of the matrix are obtained, and then to the low-frequency the component of the current matrix is sequentially applied quantization and coding with spatial prediction, then quantization, scanning and coding according to the zero-tree method are applied to the high-frequency component of the current matrix, and then entropy arithmetic coding is applied to the preliminary codes of both frequency components of the current matrix, after encoding all the source matrices the set of received codes is written to the storage device by the decoder in the process of decoding the sn At the beginning, a set of image codes is extracted from the storage device, then for each decoded color matrix, the entropy arithmetic decoding is first applied to the obtained codes, as a result of which separate codes of the low-frequency and high-frequency components of the matrix are obtained, after which spatial prediction decoding is sequentially applied to the low-frequency component of the current matrix, and dequantization, then decoding by the method of zero d is applied to the high-frequency component of the current matrix revev, dequantization and reverse scan, and then decoded by the low-frequency and high-frequency components by the inverse wavelet transform is reduced starting current matrix image, and after decoding all color matrixes restore the original image.

The disadvantage of the prototype is the low value of the compression ratio of video images having a high-frequency nature of the signal spectrum.

The objective of the technical solution is to significantly increase the compression ratio of video images with a slight decrease in the quality of the decoded image as applied to images having a high-frequency character of the signal spectrum.

The problem is solved due to the fact that in the method of encoding-decoding digital video images containing the following sequence of actions: when encoding each next image, the encoder polls the color image sensor, then the resulting image is represented by three source color matrices corresponding to the red, green and blue color components of the image, after which the source color matrices are stored in the encoder memory, then line by line at each source color matrix the wavelet transform is changed, during which the low-frequency and high-frequency components of the matrix are obtained, after which quantization and coding with spatial prediction are sequentially applied to the low-frequency component of the current matrix, then quantization, scanning and coding by the zero-tree method are applied to the high-frequency component of the current matrix, and after of this, entropy arithmetic coding is applied to the preliminary codes of both frequency components of the current matrix, after coding of all and similar matrices, the set of received codes is written to the storage device, the decoder, in the process of decoding the image, first extracts the set of image codes from the storage device, then for each decoded color matrix, the entropy arithmetic decoding is first applied to the obtained codes, as a result of which separate codes of the low-frequency and high-frequency components are obtained matrices, after which decoding with transient prediction and dequantization, then zero-tree decoding, inverse scanning and dequantization are applied to the high-frequency component of the current matrix, then the decoded low-frequency and high-frequency components use the inverse wavelet transform to restore the original current image matrix, and after decoding all color matrices, the original picture; the following differences are provided: for each row of each color matrix of the low-frequency component of the wavelet transform in the encoder after quantization, but before coding with spatial prediction, row-wise optimization is applied in accordance with the system of equations

w (x) is the input function of the wavelet transform after quantization;

q (x) is an additional function;

f (x) is the filter output function;

λ (x) is the Lagrange multiplier;

c ₁ , c ₂ - weighting factors;

a is the filter coefficient;

k is the coefficient of proportionality;

x is an argument

, λ(х), причем для кодирования используют производную суммарной функцииwith unknowns q (x),

, λ (x), and for encoding using the derivative of the total function

s (x) is the total function;

w (x) is the input function of the wavelet transform after quantization;

q (x) is an additional function;

and in the decoder, after decoding with spatial prediction, but before dequantization, filtering according to the formula

s (x) is the total function;

- функция выхода фильтра;

- filter output function;

a is the filter coefficient;

k is the coefficient of proportionality;

x is an argument

, при этом значения функции s(x) вычисляют предварительно по ее производной и с учетом краевых условий, причем и в кодере, и в декодере краевые условия выбирают одинаковыми для соответствующих строк и задают предварительно, значения коэффициентов c₁, c₂ в кодере задают предварительно, значения коэффициентов а, k для кодера и декодера принимают одинаковыми и задают предварительно, а в процессе кодирования-декодирования используют дискретные представления указанных формул, в которых аргумент x принимают в качестве номера элемента текущей строки преобразования.with unknown

, while the values of the function s (x) are preliminarily calculated according to its derivative and taking into account the boundary conditions, both in the encoder and in the decoder, the boundary conditions are chosen the same for the corresponding lines and are pre-set, the values of the coefficients c ₁ , c ₂ in the encoder are pre-set , the values of the coefficients, k for the encoder and the decoder receiving the same and is set in advance, and in the process of encoding-decoding using digital representations of said formulas in which the argument x is taken as the current item numbers element Oka conversion.

A device for implementing the proposed method for encoding / decoding digital video images consists of a SAMSUNG R530 laptop, a hama AC-150 digital web camera, a stand for a web camera, an illuminated object, an object light source, a splitter, an electricity source, and a film set. A digital webcam 2 is connected to the laptop 1, located on the stand 3, designed to capture the surface of the object 4 illuminated by the light source 5. A laptop and a light source are connected to the splitter 6, and the splitter is connected to an electricity source 7. All of the above items are located on the set 8. The laptop and the light source are in the on state, and the laptop is loaded with software for comparative analysis of the prototype and the claimed encoding methods digital video decoding. The laptop is equipped with software that allows you to implement the inventive method separately, as well as carry out an experiment to conduct a comparative analysis of video codec models (encoder and decoder) on the basis of the prototype and the inventive methods. In an experiment, by comparing two models of video codecs, the same frame is processed that is obtained programmatically from a web camera and accepted as an input image. The sensitivity of the web-camera, as well as the distance from the shooting object to the light source, are selected so that for all color components of all pixels of the input image, their relative brightness values would not exceed a discrete maximum of 254 rel. units brightness when programmatically presenting each color component with 1 byte, that is, so that there is no “flare” in the image. The experiment is organized as follows: first, the image is recorded and processed by the video codec model based on the prototype method, while the processed image is stored in the laptop’s memory, after which the saved image is processed by the video codec model based on the proposed method. All parameters and technical characteristics of the above structural elements, as well as schemes and parameters of the compared models of video codecs, ceteris paribus, are presented in tables (Table 1) and (Table 2) and in the figures (FIG. 1, FIG. 2, FIG. 3, FIG. 4).

The method of encoding-decoding digital video images described above is carried out as follows: on a laptop, a software version is launched to implement the inventive method individually or an experiment to conduct a comparative analysis of video codec models based on the prototype and inventive methods by pressing the corresponding button. After that, they expect the end of image processing and display on the screen of the results of the implementation of the proposed method separately, or the results of the experiment compared to models of video codecs using the prototype and the claimed methods.

The presence of a causal relationship between the set of essential features of the claimed object and the achieved technical result is shown in table 3. The table data is based on the results of experiments on the comparative analysis of video codec models based on the prototype and the claimed methods. Both criteria were calculated programmatically.

According to the experimental data, the proposed decoding encoding method provides an increase in the compression ratio of digital video images with a high-frequency character of the signal spectrum by an average of 8-14% with a slight decrease in the quality of the decoded image by an average of 1-2%.

The technical nature of the claimed technical solution is illustrated by the following additional materials.

FIG. 1. The structural diagram of a device for implementing the prototype and the proposed methods.

FIG. 2. Functional diagram of the encoder in the device for implementing the prototype method.

FIG. 3. The functional diagram of the encoder in the device for implementing the proposed method.

FIG 4 A schematic diagram of the optimization of coding the signal of the low-frequency component of the wavelet transform.

FIG. 5. The simulation results of the proposed method for the square sector of the low-frequency component of the wavelet transform.

FIG. 6. The simulation results of the proposed method for one line of the low-frequency component of the wavelet transform.

FIG. 7. The scheme of the experimental setup for comparing the operating efficiencies of video deck based on the prototype and the proposed methods.

FIG. 8. A photograph of an experimental setup for comparing the performance of video deck based on the prototype and the proposed methods.

An explanation of the need to introduce a combination of these distinctive features is as follows. It is known that when processing images with a high-frequency nature of the spectrum, the efficiency of the wavelet transform with respect to the compression coefficient decreases due to the fact that for such images the signal change during the transition from pixel to pixel becomes large.

So, for example, in the case under consideration, even the function of the low-frequency component of the wavelet transform w has a large first derivative in absolute values, and therefore is inconvenient for encoding.

The essence of the described method is that in the encoder another high-frequency component q is artificially mixed with the initial low-frequency wavelet transform signal w so that on the one hand the total function s = w + q becomes whiter than w (as a result, the first derivative s 'becomes convenient for compression), and on the other hand, the additional component of the high-frequency function q can be effectively suppressed on the decoding side with minimal loss in image quality when using a low-pass filter frequencies.

Data processing is carried out line by line. The presented mathematical models and the sequences of their execution by the devices of the video coding scheme (FIG. 1) are a solution to the variational problem with the target functional J

and communication restrictions

- целевой функционал; x - аргумент; b - верхняя граница области интегрирования; w(x) - входная функция; q(x) - добавочная функция;

- функция выхода фильтра; λ(x) - множитель Лагранжа; c₁, c₂ - весовые коэффициенты; а - постоянный коэффициент, характеризующий частоту среза фильтра; k - коэффициент пропорциональности, учитывающий ослабление сигнала на выходе фильтра.Where

- target functionality; x is the argument; b is the upper boundary of the integration region; w (x) is the input function; q (x) is an additional function;

- filter output function; λ (x) is the Lagrange multiplier; c ₁ , c ₂ - weighting factors; a is a constant coefficient characterizing the cutoff frequency of the filter; k is the coefficient of proportionality, taking into account the attenuation of the signal at the output of the filter.

сигналов (требование качества изображения). Приоритеты между указанными целями регулируются весовыми коэффициентами. Ограничение связи представляет собой фильтр низких частот. Оно учитывается при кодировании, а также используется при декодировании.The first term of the functional characterizes the goal of increasing the degree of smoothness of the derivative of the total function s = w + q (requirement for the amount of codes). The second term characterizes the goal of maintaining a slight difference between the original encoded w and the output decoded

signals (image quality requirement). Priorities between these goals are regulated by weighting factors. Communication restriction is a low pass filter. It is taken into account during encoding, and is also used in decoding.

кодируемого и отфильтрованного (декодируемого) сигналов.The results of a comparative analysis of the prototype and the proposed methods, shown in the figures (FIG. 5) and (FIG. 6), contain experimentally obtained characteristics for the following functions: function of the low-frequency component of the wavelet transform w, derivative of this signal w ', additional function q, total the function s equal to the sum w + q of the encoded and additional functions, its derivative s' (transmitted to the decoder), the error r of the decoded low-frequency component of the wavelet transform, defined as the difference

encoded and filtered (decoded) signals.

According to the results of the comparative analysis, it can be noted that since the total function s in the coding process becomes a smooth function, this reduces the absolute values of its first derivative s'. It becomes convenient to code the function s', since it has small absolute values, which leads to a decrease in the bit grid for these values. In addition, the function s' contains long zero sequences that are effectively encoded by an entropy encoder. As a result, the storage device requires less memory to store the encoded s', which means that the compression ratio of the image is increased.

), подавив добавочный компонент q. Поскольку действие фильтра декодера (ограничение связи) учитывается еще на стороне кодирования, то сигнал декодируемого изображения искажается слабо.At the same time, according to the initial conditions and codes of the function s', the decoder can first restore the function s itself, and then filter out the desired function w from it (in the decoder, it corresponds to the function

) by suppressing the additional component q. Since the effect of the decoder filter (communication restriction) is taken into account even on the encoding side, the signal of the decoded image is distorted slightly.

The feasibility study of the proposed method consists in the fact that when it is used, it is possible to store more images with the same amount of memory, which means that in the general case you can save on the number of such storage devices, all other things being equal. Another cost-effective application of the proposed method is its use in the framework of encoding frame video streams (digital television, online communication on-line), since a smaller amount of code requires less energy. But in this case, it is necessary to use small image formats (coding time is critical), and also have a communication channel with high noise immunity, because due to the use of derivatives, one decoding error per line generates a series of errors for the remaining elements of the line.

Claims (23)

A method of encoding / decoding digital video images, which consists in the following: when encoding each successive image, the encoder polls the color image sensor, then the resulting image is represented by three source color matrices corresponding to the red, green and blue color components of the image, after which the original color matrices are stored in the encoder memory , then a wavelet transform is applied row by row to each source color matrix, during which a low-frequency and high frequency component of the matrix, after which quantization and coding with spatial prediction are successively applied to the low-frequency component of the current matrix, then quantization, scanning, and coding are applied to the high-frequency component of the current matrix according to the zero-tree method, and then the entropy is applied to the preliminary codes of both frequency components of the current matrix arithmetic coding, after coding all the source matrices, the set of received codes is written to the storage device, Dec In the process of decoding the image, first, a set of image codes is extracted from the storage device, then for each decoded color matrix, the entropy arithmetic decoding is first applied to the obtained codes, resulting in separate codes of the low-frequency and high-frequency components of the matrix, and then to the low-frequency component of the current matrix in series apply spatial prediction decoding and dequantization, then to the high-frequency component of those For the matrix matrix, zero-tree decoding, reverse scanning and dequantization are used, after which the original current image matrix is restored using the decoded low-frequency and high-frequency components using the inverse wavelet transform, and after decoding all color matrices, the original image is restored, characterized in that for each row each color matrix of the low-frequency component of the wavelet transform in the encoder after quantization, but before coding with space With a forecast, line-by-line optimization is applied in accordance with the system of equations

w (x) is the input function of the wavelet transform after quantization; q (x) is an additional function;

- filter output function; λ (x) is the Lagrange multiplier; c ₁ , c ₂ - weighting factors; a is the filter coefficient; k is the coefficient of proportionality; x is an argument with unknowns q (x),

, λ (x), and for encoding using the derivative of the total function s' (x) = w '(x) + q' (x) s (x) is the total function; w (x) is the input function of the wavelet transform after quantization; q (x) is an additional function, and in the decoder, after decoding with spatial prediction, but before dequantization, filtering according to the formula

s (x) is the total function;

- filter output function; a is the filter coefficient; k is the coefficient of proportionality; x is an argument with unknown

, while the values of the function s (x) are preliminarily calculated according to its derivative and taking into account the boundary conditions, both in the encoder and in the decoder, the boundary conditions are chosen the same for the corresponding lines and are pre-set, the values of the coefficients c ₁ , c ₂ in the encoder are pre-set , the values of the coefficients, k for the encoder and the decoder receiving the same and is set in advance, and in the process of encoding-decoding using digital representations of said formulas in which the argument x is taken as the current item numbers element Oka conversion.

Images