RU2365994C1 - Method of detecting distortions, caused by gibbs effect, in jpeg-coding - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: invention relates to digital photography, more specifically to analysis of digital image quality. The method of detecting distortions caused by Gibbs effect in JPEG coding involves evaluation of the size of the coding unit with relative the given resolution of the printing device; determination for each coding unit, whether the size of the unit makes it distinctive to the human eye with the required printing resolution, of the approximate metric of distortion distinctiveness, caused by Gibbs effect; setting to zero corresponding elements of the approximate metric of distortion distinctiveness, caused by Gibbs effect if their values are below the preferred threshold; calculation for zero elements of the approximate metric of distortion distinctiveness, caused by Gibbs effect, of the corresponding distortion dispersion; dispersion is zeroed for the rest of the elements.

EFFECT: invention can be used in detecting distortions during JPEG coding.

4 cl, 6 dwg

Description

The invention relates to digital photography, and more particularly to the analysis of digital image quality, and may find application for the effective detection of distortion when using JPEG encoding.

A coding technique known as JPEG (the term JPEG is derived from the English name of the Joint Photographic Experts Group) was adopted as a standard (see http://www.jpeg.org/) [1], which is widely used to compress digital data forming a picture. Typically, discrete cosine transform (DCT) and quantization are used to achieve a noticeable compression ratio. In JPEG encoding, the image is divided into small blocks, usually having a size of 8 × 8 pixels, and an encoding transform, namely, discrete cosine transform (DCT), is applied to these blocks. As a result of the encoding conversion, instead of a specific number of pixels, an equivalent number of transform coefficients (DCT coefficients) is obtained. Then these coefficients are quantized.

Quantization is an operation in which DCT coefficients are divided entirely into a quantization matrix, whose value corresponds to each frequency of the region. Due to quantization, the value of the frequency component with a small DCT coefficient becomes zero. In general, energy is concentrated in the low-frequency region of the image signal, and therefore, the high-frequency component is removed by this operation.

Thus, a certain amount of information about the image is lost during the quantization process, causing the appearance of the so-called block artifacts (distortions) and artifacts caused by the Gibbs effect after reconstructing the digital image by applying the inverse coding transform to the set of quantized transform coefficients for reconstructing the encoded image. Distortions appear in the region of the boundaries of adjacent blocks in the form of an inhomogeneity (jump) in brightness, contrast, and / or color. The distortions caused by the Gibbs effect appear at sharp brightness boundaries (edges of objects) and usually are fuzzy, gray lines near the edge. With an increase in compression ratio, the manifestation of these distortions also increases.

To reduce the distortions caused by the Gibbs effect, a large number of different algorithms and methods have been proposed. However, most decisions are based on identifying edges, which is a complex and unreliable procedure, the optimal parameters of which are unknown.

The solution proposed in US patent No. 6,845,180 [1] is associated with the discovery of areas in which distortions due to the Gibbs effect are possible. The inventor came to the conclusion that these distortions can be foreseen by identifying those areas of the image that have a relatively high brightness value and high contrast. Blocks that may be subject to such distortions are detected by comparing the brightness of each pixel and its horizontal neighbor with the first threshold value T1 and the absolute value of the brightness contrast between the horizontal horizontal pixels with the second threshold value T2. If one of the brightness values exceeds the first threshold, and the difference is greater than the second threshold, the pixel is marked. The same check is carried out with a vertical neighbor. If the number of marked (marked) pixels exceeds the third threshold, the block is considered a candidate for further processing.

In US Pat. No. 6,707,952 [2], candidate blocks for further processing are selected if they have a dominant edge. To do this, use the horizontal and vertical edge masks: [-101] and [-101] ^{T. The} absolute values of the values of the pixel differences are accumulated in two variables, for the horizontal and vertical differences. The direction of the edge is determined by comparing these two values with each other, if the “horizontal” sum is greater than the “vertical”, then the presence of a horizontal edge is assumed and vice versa. The maximum of the calculated values is called the "edge content", and after comparing the "edge content" with a given smoothness threshold, it is decided whether the strongest edge is dominant or not (if yes, then the current block is processed to remove distortion).

International Patent Publication No. WO 99/22509 [3] refers to the so-called “block semaphore” and “ringing semaphore”, which determine whether the current coding block needs further processing. The block semaphore and the ping semaphore are detected by analyzing the distribution of the inverse quantization coefficients, which are the discrete cosine transform (DCT) coefficients, after the compressed binary stream undergoes inverse quantization. "Assuming that the topmost left pixel of the block among the 64 pixels making up a 8 × 8 pixel block is pixel A, the pixel to the right of pixel A is pixel B, and the pixel below pixel A is pixel C, ringing semaphore (RS) set to "I", which means that further processing is required if any of the pixels, in addition to the pixels A, B and C of the inverse quantization of the 8 × 8 pixel block, has a non-zero coefficient. " However, this simple solution provides insufficient accuracy and leads to numerous false positives.

In US Pat. No. 6,748,113 [4], the presence of distortion caused by the Gibbs effect is detected by classifying a coding block into three classes using four predefined procedures. Classification in any of the procedures consists in comparing the absolute values of the DCT coefficients in shaded areas with a certain predetermined value.

In US Pat. No. 7,050,649 [5], the presence of distortions caused by the Gibbs effect is detected by comparing a local gradient calculated using four adjacent image samples with a predetermined threshold.

In US Pat. No. 7,003,173 [6], the first edges are calculated using a Roberts filter. Then the edges are classified into those edges that need to eliminate the distortions caused by the Gibbs effect, edges that need to be sharper, or edges that don't need either, the classification is based on the “strength” of the edge and “ True Edge Threshold "(True Edge Threshold), as well as on whether the current edge is a" true visual edge "or not.

In US patent No. 6,807,317 [7] identify the first edges and smooth to remove distortion.

Various metrics for distortions caused by the Gibbs effect have been proposed in the literature; they are mainly based on edge detection and estimation of vibrations near the main edges.

In Feng, X., J.P. Allebach, "Measurement of ringing artifacts in JPEG images." Digital Publishing Edited by Allebach, Jan P .; Chao, Hui. Proceedings of the SPIE, Volume 6076, pp. 74-83 (2006). [8] it is proposed to detect false blocks containing distortions caused by the Gibbs effect by analyzing the average edge gradient in the block. The Gibbs effect distortion metrics are based on measuring local exposure as the average of the absolute value of the brightness gradients, taking into account the concept of “Barely Distinguished Differences” by Chou and Li from S. Chou and Y. Li, “A perceptually tuned sub-band image coder based on the measure of just-noticeable distortion profile, "IEEE Trans. Circuits Syst. Video Technol. 5 (6), pp. 467-476, 1995 [9].

In the work of Oguz, S.H .; Hu, Y.H .; Nguyen, TQ, "Image coding ringing artifact reduction using morphological post-filtering", Multimedia Signal Processing, 1998 IEEE Second Workshop on Volume, Issue, 7-9 Dec 1998 Page (s): 628-633 [10] proposed to use the so-called The Gibbs Effect Distortion Measurement Metric (VRM) based on averaged local variance in the region of the main edges. First, create a VRM mask by performing edge detection operations and morphological operations on the edges. Then build a local averaged dispersion around the edges, avoiding too smooth edges.

In the article by Ying-Wen Chang and Yen-Yu Chen, "Alleviating-Ringing-Artifact Filter Using Voting Scheme", ICGST International Journal on Graphics, Vision and Image Processing, Special Issue on Applicable Image Processing [11], the image is divided into smooth, edge or textured areas by applying the tree method. The way to eliminate distortions caused by the Gibbs effect in this case is based on the use of morphological operations. Voting is used to select the optimal structural element for the morphological operation.

The method described in Marziliano, P., Dufaux, F., Winkler, S., and T. Ebrahimi "Perceptual Blur and Ringing Metrics: Application to JPEG 2000", Signal Processing: Image Communication, Volume 19, No. 2, February 2004, pp.163-172 (10) [12], based on the calculation of the difference between the original and encoded images. It is proposed to determine the metric of distortions caused by the Gibbs effect for each significant vertical edge. First, vertical edges are determined in the original image (weak edges and noise are again eliminated by applying a threshold value), and the difference between the processed image and the original is calculated. Then, each row in the processed image is scanned and distortion is measured around each edge.

In a report by S. Yang, Y. Hu, T.Q. Nguyen, and D.L. Tull, "Maximum-likelihood parameter estimation for image ringing-artifact removal," IEEE Trans. Circuits Syst. Video TechnoL, vol. 11, no. 8, pp. 963–973, Aug 2001 [13], the authors propose applying k-means clustering to a block containing distortions caused by the Gibbs effect to create a planar model replacing a wavy surface with a plane to estimate the intensity of the original (uncompressed) image. The authors proceed from the assumption that the assessment of the original image is an assembly of flat surfaces. To process a wide class of images, a planar model is applied locally to small areas of the image. First of all, the number of classes is determined for various pieces of the image (for areas with texture and for areas containing only sharp edges). The authors then reduce the model to grouping in three classes for ease of calculation. This method is closest to the claimed invention.

All these methods require very high computing resources, because either they use the operation to identify the edges, or involve the use of the original (uncompressed) image.

The problem to which the claimed invention is directed is to develop such a method for detecting distortions caused by the Gibbs effect during JPEG encoding, which would make it possible to do without an edge detection operation and without performing a comparative analysis of the encoded image with the original image.

The technical result is achieved through the use of a new method, which consists in the following operations being performed:

- evaluate the size of the encoding unit in relation to a given resolution of the printing device;

- determine for each coding block, if the block size makes it visible to the human eye at the required print resolution, the approximate metric of the distinguishability of distortion caused by the Gibbs effect;

- set to zero the corresponding elements of the approximate metric of distinguishability of distortion caused by the Gibbs effect, if they have values below a predetermined threshold;

- calculate for nonzero elements of the approximate metric of distinguishability of the distortion caused by the Gibbs effect, the corresponding variance of the distortion, for the other elements the variance is considered zero (zero).

For the effective operation of the proposed method, it is important that the approximate metric of distinguishability of distortion caused by the Gibbs effect for the current image coding block is calculated based on the number of non-zero DCT coefficients, the largest index of non-zero DCT coefficients in a zigzag scan of the current image coding block and eight image coding blocks adjacent to current image encoding block.

For the effective functioning of the proposed method, it is important that the predetermined threshold is calculated based on the maximum value of the approximate metric of the distinguishability of distortion in addition to all other values.

For the effective functioning of the proposed method, it is important that the distortion variance is calculated based on the grouping (clustering) of the intensities of the decoded image.

The inventive method is applicable for both black and white, and for color images.

The novelty of the claimed invention is confirmed by the following factors:

- in the claimed invention does not apply the search for edges;

- the probability of the presence of distortion caused by the Gibbs effect is calculated in the frequency domain based on a simple and effective formula;

- the claimed method for determining the severity (strength) of distortion caused by the Gibbs effect, based on the clustering of pixel intensities.

For a better understanding of the essence of the claimed invention the following is a detailed description with the involvement of graphic materials.

Figure 1 - Scheme of the main components of a system that implements the inventive method.

Figure 2 - The main operations of the procedure for detecting distortions caused by the Gibbs effect.

Figure 3 - Procedure for computing an approximate metric of distinguishability of distortion caused by the Gibbs effect, step 202.

Figure 4 - The basis functions of the cosine transform lattice with a printed zigzag scan index on each basis function, similar to ordering a block of 8 × 8 DCT coefficients after quantization.

Figure 5 - Procedure for calculating the distortion variance, step 204.

6 - The results of the detection of distortion caused by the Gibbs effect, where the original (compressed) image is placed on top, and the distortion dispersion is imprinted on the lower image.

Figure 1 presents a block diagram of the main components of the system, with which the inventive method is implemented. The functioning of the system is controlled by a processor 101, which executes the program code recorded in the memory 102. The original black-and-white or color photo image is stored in the memory 102. The image is processed and transmitted to the display device 104. Information is exchanged via data bus 106.

Figure 2 presents the main stages of detecting distortion. At step 201, it is determined whether the encoding block of the image is visible to the human eye at a given resolution of the printing apparatus. Let the resolution of the printing device be equal to X dots / inch (usually 300-400 dpi), the pixel size of the encoding block is 8 pixels, then the width and height of the block when printing is 8 / X inches. If this size is indistinguishable by the human eye, the image is not further processed, since block distortions are also indistinguishable in this case, and no environmental distortions are detected, step 205.

Otherwise, perform the following procedures. At step 202, an approximate metric of distortion caused by the Gibbs effect is calculated for each image encoding block. Details of step 202 are provided below. At step 203, the calculated approximate metric of distortion caused by the Gibbs effect is compared with the threshold value so that all elements that are smaller than the predetermined threshold value are reset. The threshold value is defined as the maximum value among all values of the approximate metric of distinguishability of distortion caused by the Gibbs effect, multiplied by a number that is less than unity, in the preferred embodiment of the invention, this value is 0.5. At step 204, for all blocks whose corresponding approximate distortion distinguishability metric is greater than zero, the distortion variance is calculated. Details of step 204 are presented below. As a result, it is established that all blocks having nonzero dispersion are subject to distortions caused by the Gibbs effect, while the remaining blocks are not subject to these distortions.

Figure 3 presents the procedure for calculating the approximate metric of distinguishability of distortion caused by the Gibbs effect, step 202. At step 301, starting from the first encoding block of the image, calculate the approximate metric of distinguishability of distortion caused by the Gibbs effect for the current encoding block of the image.

The model of appearing distortions caused by the Gibbs effect can be formulated in accordance with the following rules.

The current block (subject to this distortion) has a high activity, that is, it is represented by a sufficient number of nonzero DCT coefficients.

At least one of its eight neighbors (top, bottom, left or right, left-top, left-bottom, right-top, right-bottom) is a smooth block (a small number of non-zero coefficients).

The current block is compressed, and the main reason for the distortion is quantization, which means that the number of zero DCT coefficients is also significant.

Consider the basic DCT features and zigzag scan indices for JPEG encoding. 4, a zigzag scan index is imprinted on each basis function, in the same way that a block of 8x8 DCT coefficients is ordered after quantization. For each 8 × 8 image block being processed, the following values were found:

1. The maximum index of the non-zero DCT coefficient, if the block is organized using a zigzag order in the vector, i _max .

2. The number of non-zero DCT coefficients N _DCT .

, k=0…7.3. The same values for its eight neighbors (adjacent blocks), 8 × 8 pixels, top, bottom, left, right, top left, bottom left, top right, bottom right.

, k = 0 ... 7.

The approximate metric of distinguishability of the distortion caused by the Gibbs effect, RM, is calculated for the current image encoding block using the following formulas:

,

,

Where

(64-N _DCT ) - the number of zero DCT coefficients in the current block. The larger the number, the higher the compression;

θ (G ₁ <0) is 1 if the block in question has the largest number of non-zero DCT coefficients among its neighbors;

θ (G ₂ <0) is 1 if the block in question has the largest index of the last non-zero DCT coefficient among its neighbors;

θ (i _max ≥5) is equal to 1 if the distortions caused by the Gibbs effect have a sufficiently high frequency.

At step 302, the inventive method proceeds to the next image encoding block. At step 303, the condition is checked whether there are any raw image blocks. If 303 gives a positive answer, the calculation of the approximate metric of distinguishability of distortion caused by the Gibbs effect is completed. Otherwise, consider the next block of the image, and steps 301-303 repeat.

Figure 5 presents the details of the procedure for calculating the distortion variance, step 204. At step 501, a check is made to see if the approximate metric of distinguishability of distortion caused by the Gibbs effect is not greater than zero. If the check in step 501 yields a negative result, the distortion variance for the current block is zeroed. Otherwise, the distortion dispersion is calculated for the current image encoding block, step 502. The distortion dispersion is calculated in order to determine the strength (severity) of the distortion caused by the Gibbs effect. It is impractical to simply calculate the variance of a potential region susceptible to distortion, since it is very likely that this region includes a distinct edge. The goal is to measure the dispersion of fuzziness (blur) near the edge. To this end, pixel intensities are clustered (grouped) in a block subject to distortion. Pixels are clustered (grouped) within such a block into several classes using the k-means algorithm. Grouping is carried out in two, three and four classes, while choosing the one with the least clustering error. The average value between the variance of the detected classes is calculated. The dispersion of the set of intensities {x ₁ ... x _N } pixels is calculated by the standard formula (<x> is the average value):

The variance between the actual pixel intensities and the “estimates for flat surfaces” obtained by clustering is calculated. As a result, the estimate of the distortion strength caused by the Gibbs effect is given as the variance in the block subject to distortion.

At 504, the method proceeds to the next image encoding block. At step 505, it is checked whether there are still raw image blocks. If there are no such blocks, then the execution of the method is completed. Otherwise, they begin to consider the next image block and repeat steps 501-505. As a result, they conclude that all blocks that have nonzero dispersion are subject to distortions caused by the Gibbs effect, and the remaining blocks are not subject to these distortions.

The proposed method is applicable to digital cameras, slide scanners, camera phones, printers or the like.

It should be borne in mind that all of the above is given as an example of the implementation of the proposed method, therefore, it should be clear to a person skilled in the art that various replacements, additions and modifications are possible without departing from the scope of protection defined in this description and in the attached claims.

Claims (4)

1. A method for detecting distortions caused by the Gibbs effect during JPEG encoding, which consists in performing the following operations:
evaluating the size of the encoding unit in relation to a given resolution of the printing device;
determine for each coding block, if the size of the block makes it visible to the human eye at the required print resolution, an approximate metric of distinguishability of distortion caused by the Gibbs effect;
set to zero the corresponding elements of the approximate distinguishability metric of distortion caused by the Gibbs effect, if they have values below a predetermined threshold;
for nonzero elements of the approximate metric of distinguishability of distortion caused by the Gibbs effect, the corresponding dispersion of the distortion is calculated, for the remaining elements the dispersion is zeroed. 2. The method according to claim 1, characterized in that the approximate metric of distinguishability of distortion caused by the Gibbs effect for the current encoding block of an image is calculated based on the number of non-zero DCT coefficients, the largest index of a non-zero DCT coefficient in a zigzag scan of the current encoding block of an image, and eight encoding blocks images adjacent to the current image encoding block. 3. The method according to claim 1, characterized in that the predetermined threshold is calculated based on the maximum value of the approximate metric of the distinguishability of distortion caused by the Gibbs effect, in addition to all other values. 4. The method according to claim 1, characterized in that the distortion variance is calculated based on the clustering of intensities of the decoded image.

Images