RU2788314C1 - Method for detecting and localizing a falsified area in jpeg images - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: invention relates to the field of information technology, namely to verifying the authenticity of images of documents of various types. The method for detecting and localizing the falsified area in JPEG images consists in the fact that the image is considered in the RGB color space, which was subjected to JPEG compression, after which, with the help of software, falsifications were performed and the modified image was saved in a lossless format, while the detection of the falsified area is based on the search for inconsistencies in structure of DCT coefficients of JPEG images, to do this, the DCT coefficients of the image are calculated with saving the information about the position of the blocks corresponding to the coefficients of 8 by 8 pixels, the quantization matrix is evaluated according to the calculated DCT coefficients, while the image is divided into several blocks, then the quantization step is estimated for each of the 64 frequencies in each block, the JPEG compression grid position is searched, the image is searched for inconsistencies of DCT coefficients to the quantization matrix, the falsified areas are localized.

EFFECT: increase in the accuracy of determining the falsified area in the image.

1 cl, 7 dwg

Description

The claimed technical solution relates to the field of authentication of images of documents of various types.

Various methods are known in the art for detecting and localizing a fake area in JPEG images (see, for example, 1. S. Ye, Q. Sun and E. Chang, "Detecting Digital Image Forgeries by Measuring Inconsistencies of Blocking Artifact," 2007 IEEE International Conference on Multimedia and Expo, 2007, pp. 12-15, doi: 10.1109/ICME.2007.4284574 [1], T. Bianchi, A. De Rosa and A. Piva, "Improved DCT coefficient analysis for forgery localization in JPEG images, "2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011, pp.2444-2447, doi: 10.1109/ICASSP.2011.5946978 [2], Weihai Li, Yuan Yuan, Nenghai Yu, Passive detection of doctored JPEG image via block artifact grid extraction, Signal Processing, Volume 89, Issue 9, 2009, Pages 1821-1829, ISSN 0165-1684, https://doi.org/10.1016/j.sigpro.2009.03.025 [3]).

However, in [1], the blocking artifact measure (BAM) is calculated from the image. In this paper, for each 8 by 8 block, the proportion of mismatched frequencies is calculated. This approach makes it possible to register inconsistent blocks even at high quality JPEG compression of the true image. In addition, in [1] another method for estimating the quantization matrix was used, the method proposed in this application makes it possible to reliably estimate the quantization matrix without fear of the influence of falsified regions.

In [2], it is assumed that the original regions will be subjected to double compression, while the fake ones will have only one compression. To search for fake areas, a statistical model is built to estimate the probability that the area was subjected to double compression. This application assumes that the spoofed image was saved in a lossless format, but the method also works if the spoofed image is saved in high quality JPEG format. In contrast to [2], in the claimed method, each 8 by 8 block is determined to be fake independently, which makes it possible to detect even small areas of falsification, which statistical methods are usually not capable of.

In [3], artifacts of JPEG compression blocks are studied. Since the discrete cosine transform runs separately in each 8 by 8 block, the boundaries between such blocks are noticeable. The pasted area will have an inconsistent mesh by assumption. The method proposed in this application uses a mathematical model of DCT coefficients, which makes it possible to determine falsified fates with a high probability, regardless of what is depicted on them, and is applicable at high compression quality of the true image, when block artifacts are hardly noticeable.

The objective of the claimed invention is to eliminate the shortcomings of the prior art. The technical result consists in providing a method for detecting and localizing a falsified area in JPEG images, which makes it possible to detect with high probability, accuracy and reliability even small areas of falsification, regardless of what is depicted on them and is applicable at a high compression quality of a true image, when artifacts blocks are barely noticeable, and also provides the ability to reliably evaluate the quantization matrix without fear of the influence of falsified areas,

The task is solved, and the claimed technical result is achieved by means of the claimed method for detecting and localizing a falsified area in JPEG images.

The claimed method is applicable in the case when an image is given and it is necessary to determine whether manipulations were performed to change the content (content) of the image (copy-paste or splicing operations) and to localize falsified areas. For example, to detect falsifications in the image of a document and, thereby, refute the authenticity of the document data.

The claimed method for detecting and localizing a falsified area in JPEG images is as follows:

An image in the RGB color space is considered. It is assumed that the true image was subjected to JPEG compression, after which, with the help of software, falsifications were performed and the modified image was saved in a lossless format.

The falsified area detection is based on the search for inconsistencies in the structure of the DCT coefficients of a JPEG image. If the image has been JPEG-compressed, then its DCT coefficients at a fixed frequency (calculated at the same grid position of 8 by 8 pixels at which JPEG compression was performed) will be distributed around values that are multiples of the quantization step q, (the number in the corresponding matrix frequency quantization at which JPEG compression was performed).

d∈[kq-1, kq+1], k∈Z

With a high probability, the DCT coefficients in the falsified region will not satisfy this property. So, for example, when copy-pasting, if the internal grid of the JPEG compression of the pasted area does not match the external grid, the DCT coefficients of the pasted area will not have non-trivial periods, despite the fact that this area was compressed with the same quality as the true image. The probability of mismatch between the outer grid and the inner one is 63/64. An illustration of a true JPEG-compressed image with a quality score of 90 is shown in FIG. 1 in FIG. 2 is an illustration of this image with a fake copy-paste done in paint.net and then saved as PNG (lossless).

Computing the DCT Coefficients of an Image

For the subsequent steps of the algorithm, it is necessary to calculate the DCT coefficients of the image while maintaining information about the position of the 8 by 8 pixel blocks corresponding to the coefficients. This is achieved by the following steps.

1. Complementing the sides of the image up to a factor of 8 with black pixels.

2. Color space transformation from RGB to YCbCr.

3. Dividing the brightness channel in the YCbCr (Y) color space into blocks of 8 by 8 pixels.

4. Applying a discrete cosine transform to each block of 8 by 8 pixels.

5. Rejection from consideration of blocks in which there are pixels with saturated values (0 or 255) in any RGB channel and blocks in which the brightness (Y) in YCbCr at the moment of transition from RBG to YCbCr went beyond the range [0, 1] . We call such pixels saturated.

Point 5 is necessary, since the DCT coefficients in such blocks can behave unpredictably and do not belong to the range [kq-1, kq+1], where k ∈ Z, q is the quantization step.

Estimation of the quantization matrix

Based on the calculated DCT coefficients, the quantization matrix is estimated. To avoid the influence of the falsified area on the matrix estimation, the image is divided into several blocks. Next, for each of the 64 frequencies in each block, the quantization step is estimated. It consists of the following steps.

1. Calculation of the factorial histogram h. h(x) - the number of DCT - coefficients that are divided by x without a remainder.

2. Normalization of h by the total number of DCT coefficients.

, для некоторого порога Т₁.3. Estimation of the quantization step as

, for some threshold Т ₁ .

конец оценивания.If

end of evaluation.

нахождение у=max{x | h(x)≥Т₂] (для некоторого Т₂). Если у<4 конец оценивания.4. In case

finding y=max{x | h(x)≥Т ₂ ] (for some Т ₂ ). If y<4 end of evaluation.

-количество DCT-коэффициентов, которые делятся на х без остатка и не принадлежат множеству М={ky±1, k∈Z}.5. Construction of a modified factorial histogram

-number of DCT coefficients that are divisible by x without remainder and do not belong to the set М={ky±1, k∈Z}.

на общее количество DCT-коэффициентов не принадлежащих множеству М.6. Normalization

on the total number of DCT coefficients that do not belong to the set M.

для некоторого порога Т₃. Если

шаг квантования оценивается как

иначе как

. В пунктах 4-7 производится попытка устранить излишний шум в гистограмме DCT-коэффициентов и произвести оценку заново. Если получается найти достаточно большой период (>3) он принимается как истинный.7. Calculation

for some threshold T ₃ . If

the quantization step is estimated as

otherwise

. In steps 4-7, an attempt is made to eliminate excessive noise in the histogram of DCT coefficients and re-estimate. If it turns out to find a sufficiently large period (> 3) it is accepted as true.

For each frequency, the quantization step mode is taken over all blocks where the step estimate is not equal to 1.

- оценка шага квантования по всему изображению для i-ой частоты,Where

- estimation of the quantization step over the entire image for the i-th frequency,

- оценка шага квантования для

-ой частоты

-го блока не равная 1, М₀ - оператор взятия моды.

- estimation of the quantization step for

th frequency

th block is not equal to 1, M ₀ is the mode-taking operator.

, то шаг квантования для i-ой частоты полагается равным 1 и не участвует при поиске несоответствий в пункте 2.4, так как множество диапазонов

в этом случае заполняет почти всю числовую ось. В случае, когда истинный шаг квантования равен 1, несмотря на то, что мода берется по неединичным значениям, полученная оценка будет меньше пяти в силу фундаментального распределения DCT-коэффициентов и не будет участвовать в дальнейшем.If

, then the quantization step for the i-th frequency is assumed to be 1 and is not involved in the search for inconsistencies in paragraph 2.4, since the set of ranges

in this case fills almost the entire number line. In the case when the true quantization step is equal to 1, despite the fact that the mode is taken over non-unit values, the resulting estimate will be less than five due to the fundamental distribution of DCT coefficients and will not participate further.

This makes it possible to evaluate the quantization matrix robustly against falsified areas, even if they occupy more than half of the image.

Finding the Position of the JPEG Compression Grid

If the estimated quantization matrix consists of a large number of ones (a small number of frequencies with found nontrivial periods in the histogram of DCT coefficients), it can be assumed that either the true image was compressed with a high quality index, or that the image was divided into blocks of 8 into The 8px (compression mesh) we chose is not the same as the split that was applied in the JPEG compression.

To exclude the second option, a JPEG compression grid position search algorithm is applied. An area of some size is taken from the image. It is chosen so that the histograms of DCT coefficients from this region are sufficiently meaningful. To do this, weakly textured areas and areas in which there are a lot of saturated pixels are excluded. There are 64 options for the position of the JPEG compression grid. For each of the possible positions, a quantization matrix is estimated as described earlier, but only for the area taken, not the entire image. The first position is accepted, in the estimated matrix of which there is a sufficient number of non-trivial values.

It should be noted that it is not reasonable from the point of view of computational costs to search for the position of the JPEG compression grid with such an algorithm initially (before completing the above steps). With a high probability, the position of the grid coincides with the naive partition indicated earlier.

Finding discrepancies between DCT coefficients and quantization matrix

- шаг квантования из оцененной матрицы квантования, соответствующий частоте.Two auxiliary images I ₁ and I ₂ are built from the values as follows. Each 8 by 8 block from the partition at the selected position of the JPEG compression grid is assigned a pixel with the same coordinates in the image I ₁ and a pixel in the image I ₂ . Dimensions I ₁ and I ₂ will match. Each pixel I ₁ contains the number of frequencies at which the DCT coefficient corresponding to this pixel-block does not belong to any range from the set

is the quantization step from the estimated quantization matrix corresponding to the frequency.

- шаг квантования из оцененной матрицы квантования, соответствующий частоте. Другими словами каждая частота из М₂ дала вклад хотя бы в один пиксель I₁. В каждый пиксель I₂ записывается количество частот из множества М₂, при которых соответствующий этому пикселю-блоку DCT-коэффициент имеет абсолютное значение больше единицы. Формируется вспомогательное изображение значений I₃ с теми же размерами, что у I₁ и I₂, по следующей формуле:A set M _{2 of} such frequencies is introduced such that at least one block at this frequency had a DCT coefficient that does not belong to any range from the set

is the quantization step from the estimated quantization matrix corresponding to the frequency. In other words, each frequency from M ₂ contributed to at least one pixel of I ₁ . Each pixel I ₂ contains the number of frequencies from the set M ₂ at which the DCT coefficient corresponding to this pixel-block has an absolute value greater than one. An auxiliary image of the values of I ₃ is formed with the same dimensions as those of I ₁ and I ₂ , according to the following formula:

где t подбирается экспериментально.

where t is chosen experimentally.

Thus, an image is obtained from values in the range [0, 1]. The value in each pixel can be interpreted as the proportion of frequencies at which the corresponding block is inconsistent, in the sense of the DCT coefficients, among those frequencies at which there are inconsistencies (M2 set).

Contrasted images I ₁ , I ₂ , I ₃ are shown in FIG. 3, fig. 4, fig. 5 respectively.

Localization of falsified areas

An auxiliary image I ₄ is formed from the I ₃ image, on which morphological opening and blurring are additionally applied to eliminate parasitic elements that arise due to random deviations of the DCT coefficients. The image I ₄ calculates the maximum - an estimate of the probability of the presence of manipulations. Image I ₄ is used to construct image I ₅ with the dimensions of the original image. The pixel value within each 8 by 8 block of image I ₅ is equal to the pixel value of image I ₄ corresponding to that block.

Estimation of the probability of the presence of manipulations and the image I ₅ - the result of the algorithm. For this example (Fig. 2), the estimate of the probability of having manipulations is 0.68302. Contrasted images I ₄ , I ₅ are shown in FIG. 6 and FIG. 7 respectively.

Thus, the above method for detecting and localizing a falsified area in JPEG images makes it possible to detect some cases of falsification of data in an image without reference to its content and has the potential to be used to verify the authenticity of images of documents of various types.

Claims (19)

A method for detecting and localizing a falsified area in JPEG images, which consists in considering an image in the RGB color space that has been subjected to JPEG compression, after which falsifications were performed using software and the modified image was saved in a lossless format , characterized in that the detection of a falsified area is based on the search for inconsistencies in the structure of the DCT coefficients of a JPEG image, for this: - calculate the DCT coefficients of the image with saving information about the position of the blocks corresponding to the coefficients of 8 by 8 pixels: complement the sides of the image up to a multiplicity of 8 black pixels; transform the color space from RGB to YCbCr; produce a partition of the brightness channel in the color space YCbCr (Y) into blocks of 8 by 8 pixels; applying a discrete cosine transform to each block of 8 by 8 pixels; blocks are discarded from consideration, in which there are pixels with saturated values (0 or 255) in any RGB channel and blocks in which the brightness (Y) in YCbCr at the moment of transition from RBG to YCbCr went beyond the range [0, 1]; - the quantization matrix is estimated by the calculated DCT coefficients, while the image is divided into several blocks, then for each of the 64 frequencies in each block the quantization step is estimated: calculate the factorial histogram h, h(x) - the number of DCT coefficients that are divided by x without a remainder; normalize h by the total number of DCT coefficients; estimate the quantization step as

for some threshold Т ₁ if

end of assessment when

finding y=max{x|h(x)≥T ₂ }, for some threshold T ₂ if y<4 end of evaluation; construct a modified factorial histogram

- the number of DCT coefficients that are divisible by x without a remainder and do not belong to the set М={ky±1, k∈Z}; produce normalization

on the total number of DCT coefficients that do not belong to the set M; make a calculation

for some threshold Т ₃ if

, the quantization step is estimated as

otherwise

then, for each frequency, the quantization step mode is taken over all blocks, where the step estimate is not equal to 1; - search for the position of the JPEG compression grid: - search for inconsistencies DCT coefficients quantization matrix; - produce localization falsified areas.

Images