RU2614576C1 - Method for encoding images based on nonlinear forming system - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: method for encoding images is provided based on the nonlinear forming system, wherein the said unit is divided into a plurality of small areas at the stage of forming the prediction signal, and the prediction signal of the non-square small area is formed using the signal of the already reconstructed pixel or the unit of pixels, which contact with the first side. The processed unit is identified at the stage of the compressed signal forming, depending on the range of brightness and contrast belonging to the high contrast or low contrast image, transforming of the said unit is performed to the identified low contrast image in the sequence of pixels, estimating parameters of the nonlinear forming system, the formation the compressed signal is performed by encoding parameters of the nonlinear forming system, the formation of the compressed signal is performed for the identified high contrast image by encoding the residual signal, the compressed signal is decompressed and the decompressed signal is stored as the restored pixel signal.

EFFECT: improving the quality of the reconstructed video information.

3 dwg

Description

The invention relates to computing, and in particular to the field of video processing.

The importance of information support for the activities of government officials determines the need to expand the list and improve the quality of the information and communication services provided to them. At the same time, the visual presentation of information is the most informative and convenient, providing such information about control objects that cannot be obtained by other types and methods of information exchange. A characteristic feature of video information is the presence of various types of redundancy. When eliminating redundancy of video information, methods that take into account linear relationships between elements are mainly used.

The known method (Recommendation ITU-T H.264. Advanced video coding for generic audiovisual services (Series H: Audiovisual and multimedia systems. Infrastructure of audiovisual services - Coding of moving video) - 2012 - 01 - 01. - ITU, 2012. - 680 s.), According to which macroblocks with a size of 16 × 16 points of the processed image frame are taken, a macroblock forecast is formed from samples of the current frame previously encoded and restored (INTRA mode), or taking into account changes that have occurred in the current frame compared to one or several previous or subsequent (INTER-mode), calculate the macroblock residual coefficients by subtracting the obtained forecast from the current macroblock, perform quasi-orthogonal frequency conversion of the residual coefficients, quantize the set of frequency coefficients obtained on the basis of the frequency conversion, while the set of converted and quantized coefficients serves as the initial for the reverse data reconstruction channel, then they build the coefficients in a linear array in ascending order and produce entropy coding of an ordered array of coefficients, while forming the output bit sequence encoder reliability from encoded coefficients and additional information required for correct macroblock decoding (prediction mode, quantization coefficient, etc.).

The disadvantage of this method is the insufficiently high quality of the restored video information when encoding low-contrast images and images with a small dynamic range (d≤100) [Stremoukhov, M.V. Problems of efficient coding of images with low contrast and narrow dynamic range. / M.V. Stremoukhov, S.V. Chistyakov, I.V. Furtsev // Actual directions of development of security systems, special communications and information for the needs of public administration: IX All-Russian Interdepartmental Scientific Conference (Orel, February 11-12, 2015). At 12 h. Part 9. / Under the general. ed. V.V. Mizerova. - Eagle: Academy of the Federal Security Service of Russia, 2015. - 163 p., S. 148-151].

The closest in technical essence to the claimed method and selected as a prototype is a method for predictive encoding of images (patent RU 2526764 from 12.25.2009), which consists in dividing the input image into many blocks, generating a prediction signal relative to the pixel signal, which is included into the block under consideration, which must be processed, from a plurality of blocks, a residual signal is formed between the pixel signal of the block under consideration and the generated prediction signal, when ohm, at the stage of generating the forecast signal, the block under consideration is divided into many small areas, at least one of the small areas is non-square and the length of the first side of the non-square small area exceeds the length of the second side, which differs from the first side, and at the stage of generating the forecast signal squared small area using the signal of the already restored pixel, which is in contact with the first side, form compressed th signal by encoding the residual signal, decompress the compressed signal and save the decompressed signal as a reconstructed pixel signal.

The disadvantage of the prototype method is the insufficiently high quality of the restored video information at the reception when encoding images containing small contrasting elements (inscriptions), low-contrast images, images obtained in twilight conditions, low light, fog and smoke, defined by the following:

1) the use of algorithms based on two-dimensional cosine transform (DCT), with standard procedures for quantizing its coefficients, leads to a significant decrease in the overall subjective quality of the reconstructed video information at fairly high values of the objective indicator by the criterion of peak signal to noise ratio (SNR);

2) the lack of effective coding procedures that take into account non-linear relationships between image elements with low contrast or narrow dynamic range.

Given the existing requirements for the quality of the video information being restored at the reception, more thorough processing is necessary. Under these conditions, methods that do not take into account non-linear relationships between image elements with low contrast or narrow dynamic range become unacceptable and do not provide sufficient quality of the restored video information.

This circumstance determines the need to develop ways to improve the quality of the restored video information by improving the visual perception of images with low contrast or a narrow dynamic range, by adding procedures that take into account non-linear relationships between the elements of such images with effective coding.

The objective of the invention is to develop a method of encoding images based on a nonlinear forming system, which allows to increase the visual subjective quality of the restored video information in comparison with the known method of frequency conversion of DCT at sufficiently comparable values of the objective indicator by the criterion of SEQ.

In the claimed method, this problem is solved in that in the known method for predictive encoding of images, which consists in dividing the input image into a plurality of blocks, generating a prediction signal for a pixel signal, which is included in the considered block to be processed, from a plurality of blocks, form a residual signal between the pixel signal of the considered block and the generated prediction signal, while at the stage of generating the prediction signal, the ra a block being considered into a plurality of small areas, at least one of the small areas being non-square and the length of the first side of the non-square small area exceeds the length of the second side, which is different from the first side, and at the stage of generating the prediction signal, a prediction signal of the non-square small area is generated using the signal already restored pixel, which is in contact with the first side, in addition, at the stage of generating the compressed signal, we identify the processed the second block for belonging to a low-contrast or high-contrast image, preliminary processing (sweep) is carried out for a low-contrast image, the parameters of the nonlinear forming system are evaluated, a compressed signal is generated by encoding the parameters of the non-linear forming system, a compressed signal is generated for the high-contrast image by encoding the residual signal, the compressed signal is unpacked and storing the decompressed signal as a reconstructed pixel signal.

A new set of essential features allows you to achieve the specified technical result due to the fact that the processed block is identified as belonging to a low-contrast or high-contrast image, preliminary processing (sweep) is performed for a low-contrast image, the parameters of the nonlinear forming system are evaluated, a compressed signal is generated by encoding the parameters of the nonlinear forming system form a compressed signal for high-contrast image by means of encoded After extracting the residual signal, decompress the compressed signal and save the decompressed signal as a reconstructed pixel signal.

The analysis of the prior art made it possible to establish that there are no analogues that are characterized by a combination of features that are identical to all the features of the claimed method of encoding images based on a nonlinear forming system. Therefore, the claimed invention meets the condition of patentability "novelty."

Search results for known solutions in this and related fields of technology in order to identify features that match the distinctive features of the claimed object from the prototype showed that they do not follow explicitly from the prior art. The prior art also did not reveal the popularity of the impact provided by the essential features of the claimed invention, the transformations on the achievement of the specified technical result. Therefore, the claimed invention meets the condition of patentability "inventive step".

The claimed invention is illustrated by the following drawings:

FIG. 1 - an embodiment of a system for efficient coding of video information, providing improved perception of the restored images according to the proposed method;

FIG. 2 - stages of effective coding of video information according to the proposed method;

FIG. 3 - the results of assessing the quality of perception of the reconstructed images in devices for efficient coding of video information.

Implementation of the inventive method includes the steps of: splitting an input image into a plurality of blocks, further splitting a block into a plurality of small areas, wherein at least one of the small areas is non-square and the length of the first side of the non-square small area exceeds the length of the second side, which differs from the first parties; generating prediction signals with respect to the pixel signal included in the considered small area; generating a residual signal between the pixel signals of the considered small area and the generated prediction signal; generating a compressed signal by encoding a residual signal; and generating compressed image data, which includes a compressed signal and information related to generating prediction signals, indicating a method for generating a prediction signal of a small area in predictive coding; extracting the residual signal of a small area from the compressed image data and decompressing the reconstructed residual signal; extracting information related to the generation of prediction signals from the compressed image data; and generating a prediction signal of a small area based on information associated with the generation of prediction signals; decompressing the pixel signal of the small region by adding the reconstructed residual signal to the prediction signal of the small region; and storing the decompressed pixel signal as a reconstructed pixel signal.

In addition, at the stage of generating a compressed signal, a residual signal with narrow ranges of brightness and contrast (d≤100) [Stremoukhov, M.V. Problems of efficient coding of images with low contrast and narrow dynamic range. / M.V. Stremoukhov, S.V. Chistyakov, I.V. Furtsev // Actual directions of development of security systems, special communications and information for the needs of public administration: IX All-Russian Interdepartmental Scientific Conference (Orel, February 11-12, 2015). At 12 h. Part 9. / Under the general. ed. V.V. Mizerova. - Eagle: Academy of the Federal Security Service of Russia, 2015. - 163 p., S. 148-151] can be attributed to images with low contrast or narrow dynamic range.

In addition, a compressed signal for the specified residual signal can be obtained by pre-processing it, evaluating the parameters and coding the parameters of the nonlinear forming system.

In addition, at the stage of generating the compressed image data, information related to the generation of the compressed signal indicating the method of generating the compressed signal depending on the brightness and contrast can be encoded and the signal obtained by encoding can be output together with the compressed signal.

Description of the image encoding method based on a non-linear forming system.

The set of functional blocks and the system of logical connections between them that implement the claimed encoding method are presented in FIG. one.

FIG. 2 is a flowchart illustrating a method of encoding images based on a non-linear shaping system according to an embodiment of the invention.

In particular, as described above, FIG. 2 illustrates a coding process based on a non-linear generating system for each block of block-divided image.

At step 201 of FIG. 2 register the image [Fisenko, V.T. Computer processing and image recognition: Textbook. allowance. / V.T. Fisenko, T.Yu. Fisenko. - St. Petersburg: St. Petersburg State University ITMO, 2008. - 192 p., S. 3-11, 31-45], which is block-wise fed to the input of the coding system as input data.

At step 202, the image block having N × N pixels is subdivided into a plurality of small areas using a divider into blocks 101 of FIG. 1a. In this embodiment, although the number N is 8 (N = 8), it may be equal to another integer. A small area in an embodiment of the prototype is shown as having a non-square shape.

Then, at step 203, the initial setting of the value t = 1 of the counter t, which identifies the small areas to be processed, is performed, after which the processes at steps 204-215, which should be described below, are performed relative to the corresponding small areas determined by the counter t, which after of each cycle of steps 204-215 grows by "1" up to the value t = N. In this case, first, at step 204, a plurality of variants of prediction signals are generated with respect to the small area under consideration (for example, at (t = 1), the first small area) using the prediction signal generator 103. In the prototype embodiment, at step 205, a prediction signal for the first small region is generated by extrapolating an already restored pixel (the pixel signal of a group of already restored pixels is a pixel signal, which is encoded and then decompressed earlier and stored in memory frame monitor 108), which is in contact with the small area under consideration, by selecting one of nine methods for extrapolating the average value of the already restored pixels, which gives the smallest error among the prediction signal variants obtained by the nine indicated methods as the prediction signal, and determines the method for generating a certain signal prediction as a signal of the prediction mode (hereinafter referred to as “associated with the formation of prediction signals and formation "). For subsequent small regions, a small region having the smallest error of the nine variants of the prediction signals obtained by the nine indicated methods is used as a prediction signal. The above steps 202-203 can be performed in accordance with the description presented in the prototype [Pat. 2526764 Russian Federation, IPC ⁷ H04N 19/50. A device, method and program for predictive image encoding, a device, method and program for predictive image decoding, and an encoding / decoding system and method. / Boon C.S. Suzuki Yoshinori, Tan T.K .; Applicant and patent holder NTT DOCOMO, INC. (JP) No. 20111134659/08; declared 12/25/09; publ. 08/27/14, Bull. Number 24. - 42 p.: Ill., S. 13, 14, 16, 17]. Then, step 206 is performed, in which a difference is obtained between the pixel signal of the considered tth small region and the prediction signal of this small region (hereinafter referred to as the "residual signal between the pixel signals of the considered small region and the generated prediction signal", or "residual signal", or "difference image") using the subtraction module 102, characterized in that the difference between the pixel signal of the considered t-th small area and the predicted signal is analyzed This small area determines the method of generating a certain signal of the method of generating a compressed signal depending on the brightness and contrast, as the signal of the method of generating a compressed signal (hereinafter referred to as “information related to the formation of a compressed signal”), depending on the range of brightness and contrast refers to ordinary images, then using the block switching coding modes 104 provide the execution of step 207: frequency conversion of the resulting residual th signal converter 105, the subsequent quantization of the transformed residual signal in block 109 and entropy coding the quantized residual signal via a variable length code or arithmetic code at block 113, forming thereby the compressed signal [Pat. 2526764 Russian Federation, IPC ⁷ H04N 19/50. A device, method and program for predictive image encoding, a device, method and program for predictive image decoding, and an encoding / decoding system and method. / Boon C.S., Suzuki Yoshinori, Tan T.K .; Applicant and patent holder NTT DOCOMO, INC. (JP) No. 20111134659/08; declared 12/25/09; publ. 08/27/14, Bull. Number 24. - 42 pp., Ill., S. 14, 17], or to images with low contrast or a narrow dynamic range [Stremoukhov, M.V. Problems of efficient coding of images with low contrast and narrow dynamic range. / M.V. Stremoukhov, S.V. Chistyakov, I.V. Furtsev // Actual directions of development of security systems, special communications and information for the needs of public administration: IX All-Russian Interdepartmental Scientific Conference (Orel, February 11-12, 2015). At 12 h. Part 9. / Under the general. ed. B.V. Mizerova. - Eagle: Academy of the Federal Security Service of Russia, 2015. - 163 p., S. 148-151]. In the latter case, the encoding mode switch 104 provides: at step 208, converting the tth small region under consideration into a sequence of pixels (scan) using the preprocessing unit 107 (using, for example, some recursive scan [Alexandrov, V.V. Presentation and image processing: Recursive approach. / VV Aleksandrov, ND Gorsky. - L .: Nauka, 1985. - 192 p., P. 38-44]) and at step 209 - evaluation of the parameters of the nonlinear forming system block 106 by forming from the last the sequence of pixels of the current encoded subunit with the simultaneous formation of an exciting sequence corresponding to the given subunit (step 210), which provide identification of the parameters of the nonlinear forming system (step 211), which is a parametric model of the encoded subunit [Stremoukhov, M.V. A mathematical model of the forming system, taking into account non-linear statistical dependencies of images. / M.V. Stremoukhov, S.V. Chistyakov // News of SFU. Technical science. - 2012. - No. 6. - S. 156-161]; then, at step 213, subsequent quantization in block 109 of the obtained values of the parameters of the nonlinear forming system and entropy coding in block 113 of the quantized values of the parameters of the nonlinear forming system using a variable-length code or arithmetic code, also generating a compressed signal in accordance with the procedures provided for by standard N. 264 [Guk, I. Features of video compression according to the recommendation of H.264 / MPEG 4 Part 10 / I. Guk // Components and technologies. - 2006. - No. 2. - S. 5-7. - Guk_2006_02_20.pdf. Source: http://www.kit-e.ru]. Then, in the next step 213, the compressed signal and the information obtained in steps 207 or 212 associated with the generation of the prediction signals and related to the generation of the compressed signal are output from the output terminal 114.

The encoded signal of the tth small region is used to predict the t + 1 small region to be processed next. As a result of this, the residual signal of the tth small region is decanted by means of a dequantifier 110, is inverted by an inverter 111, and, depending on the information generated by the generation of the compressed signal, the encoding mode switching unit 104 (Fig. 1b) selects the corresponding inverse transformation mode (using block 111.3 for ordinary images or with blocks 111.1 and 111.2 for images with low contrast or narrow dynamic range, while block 111.1 could l is implemented in accordance with the already known solution (patent No. 125749 dated 03/10/2013), and block 111.2 implements the procedures opposite to that performed by block 107 (similar to those described in the literature [Alexandrov, VV Presentation and image processing: Recursive approach. / V .V. Aleksandrov, ND Gorsky. - L .: Nauka, 1985. - 192 p., S. 38-44]), and then, it is added to the prediction signal obtained as described above by the adder 112 to generate the reconstructed signal of the t-th small region (step 214). Then, in order to generate a prediction signal of the next t + 1 small region, the reconstructed signal of the t-th small region is temporarily stored in the frame memory 108 (step 215).

The processes in steps 204-215 are performed relative to the respective blocks in question, determined by the counter t, and when it is determined that the processing of all blocks in question is completed (t = N), in step 203, the process of FIG. 2 ends.

The claimed method of encoding images based on a nonlinear forming system provides an increase in the visual subjective quality of video information recovery compared to the well-known method of frequency conversion of DCT at comparable values of the objective indicator by the criterion of the peak signal to noise ratio (SNR). To prove the achievement of the claimed technical result, experimental studies are presented in accordance with ITU-T Recommendation VT.500-13 with a limited scope for the "home environment" by a group of up to 15 observers who are not experts.

To determine the subjective quality assessment, test sets of ordinary images and images that are low-contrast and have a narrow dynamic range were used.

These images were processed using the embodiment according to the prototype method and the implementation of the image encoding based on the nonlinear forming system (Fig. 1) according to the proposed method.

The evaluation results according to the prototype method and the proposed method (Fig. 3) indicate an increase in the visual subjective quality of video information recovery compared to the known method of frequency conversion of DCT at sufficiently comparable values of the objective indicator according to the peak signal to noise ratio (SNR) in the claimed method and the solution of the problem of the invention.

Claims (1)

A method for encoding images based on a nonlinear forming system, comprising the steps of dividing the input image into a plurality of blocks, generating a prediction signal for a pixel signal that is included in the considered block to be processed, from the plurality of blocks, generating a residual signal between the pixel signal of the considered block and the generated prediction signal, while at the stage of generating the forecast signal, the block under consideration is divided into a plurality of sky smaller areas, at least one of the small areas is non-square and the length of the first side of the non-square small area exceeds the length of the second side, which is different from the first side, and at the stage of generating the prediction signal, a prediction signal of the non-square small region is generated using the signal of the already restored pixel or block of pixels that are in contact with the first side, characterized in that at the stage of forming the compressed signal identify the processed block in depending on the range of brightness and contrast values for belonging to a high-contrast or low contrast or narrow dynamic range image, for the image identified as having a low contrast or narrow dynamic range, the block under consideration is converted into a sequence of pixels, the parameters of the nonlinear forming system are evaluated, and a compressed signal is generated by encoding the parameters of the nonlinear forming system, carry out for a high-contrast image identified as generating a compressed signal by encoding a residual signal, decompress the compressed signal and save the decompressed signal as a reconstructed pixel signal.

Images