CN109919927B - Multi-object tampering detection method based on rapid quaternion polar harmonic transformation - Google Patents

Abstract

The invention belongs to the technical field of digital image authentication, and particularly relates to a multi-object tampering detection method based on fast quaternion polar harmonic transformation. The invention provides a multi-object tampering detection method based on fast quaternion polar harmonic transformation, which comprises the steps of firstly obtaining a functional relation between an entropy rate and a gFAST characteristic point extraction threshold value through learning, determining an adaptive threshold value by using the function and extracting a gFAST characteristic point; then, constructing local features of the image through an FQPCET algorithm; then, obtaining a matching result by using a kNN-based feature matching algorithm; and finally, performing k-means clustering on the offset of the matching result, and performing RANSAC, ZNCC and morphological processing on various types to obtain a final positioning result. The experimental result shows that the method not only can effectively detect the multi-copy-paste tampering, but also has good robustness to conventional signal attack and geometric attack, and the time cost of the algorithm is very low.

Description

Multi-object tampering detection method based on rapid quaternion polar harmonic transformation

Technical Field

The invention belongs to the technical field of digital image authentication, and particularly relates to a multi-object tampering detection method based on fast quaternion polar harmonic transformation.

Background

Digital images are important sources of information and play an indispensable role in the fields of mass media, litigation testimony, academic research and the like. But some behaviors that maliciously tamper with the image semantics cause the reliability of the digital image to be reduced; among them, copy-paste tampering (CMF) is a more common form. CMF copies one or more regions in an image into the same image for the purpose of hiding, duplicating, or moving semantic components in the image. With the development and popularity of image processing software (e.g., PS, etc.), such tampering is also easy to implement for those without technical background. However, since the source and target regions are both from the same image, the various statistical features are similar, and the tampered image may be subject to both conventional signature and geometric attacks, making detection of CMFs relatively difficult.

In recent years, many researchers have invested time and resources in the detection and localization algorithms of CMFs. The proposed method is mostly based on the following three steps: feature extraction: calculating the proper characteristics of each pixel point or specific partial key points to represent the field of the pixel points; matching: measuring the similarity degree of any two characteristics by using a certain distance evaluation index, and classifying the characteristics which are close to each other into one class; and (3) post-treatment: and accurately positioning the actual boundary of the CMF by using the matching result obtained in the last step. The detection techniques of CMF can be divided into two categories according to whether the local features of the image are extracted from each pixel or only from some selected key points: block-based methods and feature point-based methods.

The block-based method divides the image to be detected into several regular blocks, which are then used to construct feature vectors. Although this type of method can detect tampering, especially small and smooth areas, with high accuracy without attack, the time complexity of this type of method is large and it cannot effectively resist large-scale geometric attacks. Feature point-based methods detect some specific keypoints from the image and then use the neighborhood of these keypoints to construct a feature vector. Since the number of feature points is often much smaller than the total number of pixels of an image, this type of method has certain advantages in terms of time and space complexity. However, the method based on feature points sometimes cannot accurately locate the tampered area, and is mainly limited by the following three aspects: firstly, most of feature point extraction algorithms cannot extract uniform key points in a small region or a smooth region; second, feature point features do not perform well in terms of robustness and distinctiveness; third, most post-processing algorithms can only estimate a single model from a set of matching points, and therefore cannot effectively detect multiple copy-paste tampering.

Disclosure of Invention

The invention provides a multi-object tampering detection method based on fast quaternion polar harmonic transformation, aiming at solving the technical problems existing in the existing copying and pasting tampering detection technology.

In order to solve the technical problems, the invention adopts the following technical scheme:

a, step a: acquiring an image I to be detected, and setting thresholds C =0.1, delta =0.01, N =5 and M =4N; wherein C is an ideal feature point density threshold; Δ is a maximum deviation threshold; n represents the neighborhood diameter of the feature point; m represents the fast quaternion polar harmonic transform FQPCET upsampling parameters;

step b: training to obtain image set (TP) _i An entropy-threshold fitting function T = f (E);

wherein, the step b specifically comprises the following steps:

step b.1: executing SLIC algorithm on any image in FAU library to obtain training image set { TP } _i }；

Step b.2: calculating TP _i Entropy E of _i Obtaining an entropy description vector E = { E = } _i }；

Step b.3: extraction of TP _i The characteristic points of (1); calculating a point-extracting threshold parameter T when the density X of the characteristic points belongs to (C-delta, C + delta) by using a dichotomy _i Obtaining a threshold description vector T = { T = { (T) } _i }；

Step b.4: performing function fitting by using E and T to obtain an entropy-threshold fitting function T = f (E);

step c: self-adaptive gFAST feature point detection;

wherein, the step c specifically comprises the following steps:

step c.1: executing SLIC block partitioning algorithm on an image I to be detected to obtain a non-overlapping image block set B = { B = _i }；

Step c.2: using 8 pre-defined filtering templates for an image I to be detected

Filtering to obtain 8 filtered images

Step c.3: whether any pixel (x, y) in the image I to be detected meets (x, y) epsilon B or not is examined _i (ii) a Wherein, (x, y) refers to cartesian coordinates of the pixel points; if so, calculate B _i Global entropy E of _i And then obtaining a lifting point threshold value T _i ＝f(E _i )；

Step c.4: survey vector

If the investigation vector contains 5 continuous elements which are connected end to end and are all larger than T _i Then (x, y) is marked as gFAST feature point; and traversing all pixels to obtain a preliminary gFAST feature point set X' = { X _i ＝(x,y) _i }；

Step c.5: executing an NMS algorithm on the X' to obtain a gFAST key point set X;

step d: extracting FQPCET characteristics;

wherein, the step d specifically comprises:

step d.1: determining any element X in a set X _i Circular neighborhood O of _i ；

Step d.2: from O _i Obtaining the red, green and blue color channel components f _R [i,j]、f _G [i,j]And f _B [i,j]Wherein, (i, j) refers to pixel point location;

and performing upsampling to obtain f [ u, v ], (u, v) representing the position of the upsampled pixel; :

step d.3: by calculating f [ u, v ]]Fast two-dimensional Fourier transform to obtain polar harmonic transform PCET coefficient matrix M _nl ：

Wherein the content of the first and second substances,

representing a fast two-dimensional fourier transform;

step d.4: will f is mixed _R [i,j]、f _G [i,j]And f _B [i,j]Calculating the obtained polar harmonic transformation PCET coefficient matrix

Polar harmonic transformation PCET coefficient matrix

Sum-polar harmonic transformation PCET coefficient matrix

Substituting the formula to obtain a quaternion polar harmonic transformation QPCET coefficient matrix

Step d.5: QPCET coefficient matrix for quaternion polar harmonic transformation

And (3) selecting characteristics to form a characteristic vector f:

step d.6: traversing the feature point set X and aiming at any feature point X _i Step d.1-step d.5 are performed, get the characteristic direction set of quantities F = { F _i Get the coordinate-feature set S = { S = } _i ＝{x _i ,f _i }}；

Step e: performing kNN-based feature matching on the coordinate-feature set S to obtain a matching point set P = { (S, S') _i }；

Step f: clustering post-processing based on k-means;

wherein, the step f specifically comprises:

step f.1: for matching point set P = { (s, s') _i Performing offset vector calculation to obtain an offset vector set Dt:

Dt＝{(Δx,Δy) _i }＝{(x-x′) _i }；

step f.2: calculating the length and direction of any offset vector in Dt to obtain a length set Le and a direction set Or:

step f.3: carrying out histogram statistics on Le and Or to obtain an offset vector set Dt' after primary screening;

step f.4: performing k-means clustering on Dt' to obtain k classes { C ₁ ,C ₂ ,…,C _k }；

Step f.5: traversing all the clustering results, and clustering C to any one of the clustering results _i RANSAC, ZNCC and morphological processing operations are sequentially executed, and the union of the obtained results is used as a final detection result.

The invention provides a multi-object tampering detection method based on fast quaternion polar harmonic transformation, which comprises the steps of firstly obtaining a functional relation between an entropy rate and a gFAST characteristic point extraction threshold value through learning, determining an adaptive threshold value by using the function and extracting a gFAST characteristic point; then, constructing local features of the image through an FQPCET algorithm; then, obtaining a matching result by using a characteristic matching algorithm based on kNN; and finally, performing k-means clustering on the offset of the matching result, and performing RANSAC, ZNCC and morphological processing on various types to obtain a final positioning result. The experimental result shows that the method not only can effectively detect the multi-copy-paste tampering, but also has good robustness to conventional signal attack and geometric attack, and the time cost of the algorithm is very low.

Compared with the prior art, the invention has the following beneficial effects:

firstly, an entropy-threshold function fitting algorithm is provided and adopted, the method can self-adaptively determine a point-extracting threshold value, and ensure that the global density of the feature points is uniform, so that the tampering of a small area or a smooth area can be effectively detected;

secondly, a new key point detection method gFAST is provided and adopted, the method is robust to various geometric attacks, the time complexity is small, and the algorithm speed and the geometric attack robustness are improved;

thirdly, a new moment calculation method FQPCET is provided and adopted, and compared with the original calculation method, the method has obvious advantages in the aspects of calculation speed, numerical stability and calculation precision, can effectively reflect the color information of the image and the correlation among color channels, and improves the detection precision and the algorithm speed of the method.

Drawings

Fig. 1 is a schematic flowchart of a multi-object tampering detection method based on fast quaternion polar harmonic transformation according to the present invention;

FIGS. 2 (a) -2 (c) are exemplary images used by the detection method of the present invention; wherein, fig. 2 (a) is an original image; FIG. 2 (b) is a tampered image; FIG. 2 (c) is a tampered area;

FIGS. 3 (a) -3 (l) are schematic diagrams illustrating the effect of the detection process of the present invention; wherein, fig. 3 (a) is an example of performing a SLIC blocking algorithm; fig. 3 (b) is an example of extracted adaptive gFAST feature points; FIG. 3 (c) is a characteristic representation; fig. 3 (d) and fig. 3 (e) are schematic diagrams of the matching pair and the correct matching pair, respectively; fig. 3 (f) and 3 (g) are schematic diagrams of offset vector cluster 1 and offset vector cluster 2, respectively; fig. 3 (h), fig. 3 (i), fig. 3 (j), and fig. 3 (k) are schematic diagrams of a cluster positioning result 1, a cluster positioning result 2, a cluster positioning result 3, and an overlay result, respectively; FIG. 3 (l) shows the result of marking the resulting tampered area;

fig. 4 (a) -4 (d) are diagrams of examples of locating tampered areas in the FAU image library; fig. 4 (a) is a diagram of a tampered region locating sample of a fisherman image in the FAU image library; fig. 4 (b) is a diagram of a tampered region location sample of a motorcycle image in the FAU image library; FIG. 4 (c) is a sample graph of tampered areas of a fricks image in the FAU image library; FIG. 4 (d) is a diagram of a tampered region locating sample of mask images in the FAU image library;

5 (a) -5 (d) are diagrams of examples of locating tampered areas in the GRIP image library; fig. 5 (a) is a diagram of a tampered region positioning sample of TP _ C01_002 in the GRIP image library; FIG. 5 (b) is a diagram of a positioning sample of tampered region of TP _ C01_011 in GRIP image library; FIG. 5 (C) is a diagram of a tampered region location sample of TP _ C02_001 in GRIP image library; fig. 5 (d) is a diagram of a tampered region location sample of TP _ C02_047 in the GRIP image library;

6 (a) -6 (d) a graph of the comparison result of the robustness of the embodiment of the invention and the comparison method in the FAU image library;

7 (a) -7 (d) graphs comparing robustness of the embodiment of the present invention and the comparison method in the GRIP image library.

Detailed Description

The invention provides a multi-object tampering detection method based on fast quaternion polar harmonic transformation, aiming at solving the technical problems existing in the existing copying and pasting tampering detection technology.

The multi-object tampering detection method based on the rapid quaternion polar harmonic transformation comprises the following steps: as shown in fig. 1, step a: acquiring an image I to be detected, and setting thresholds C =0.1, delta =0.01, N =5 and M =4N; wherein C is an ideal characteristic point density threshold; Δ is a maximum deviation threshold; n represents the neighborhood diameter of the feature point; m represents the fast quaternion polar harmonic transform FQPCET upsampling parameters;

specifically, the experimental images provided in fig. 2 (a) -2 (c) are used as examples for explanation; fig. 2 (a) shows an original image, fig. 2 (b) shows a tampered image, and fig. 2 (c) shows a tampered area.

On the basis of the completion of step a, continuing to implement step b: training to obtain image set (TP) _i An entropy-threshold fitting function T = f (E);

wherein, step b specifically includes:

step b.1: executing SLIC algorithm on any image in FAU library to obtain training image set (TP) _i }；

Step b.2: calculating TP _i Entropy E of _i Obtaining an entropy description vector E = { E = } _i }；

Step b.3: extraction of TP _i The characteristic points of (1); solving a point-extracting threshold parameter T when the density X of the characteristic points belongs to (C-delta, C + delta) by utilizing a dichotomy _i Obtaining a threshold description vector T = { T = { (T) } _i }；

Step b.4: performing function fitting by using E and T to obtain an entropy-threshold fitting function T = f (E);

on the basis of the completion of step b, continuing to perform step c: self-adaptive gFAST feature point detection;

the step c specifically comprises the following steps:

step c.1: executing SLIC block partitioning algorithm on an image I to be detected to obtain a non-overlapping image block set B = { B = _i }; fig. 3 (a) is an example of the SLIC block partitioning algorithm performed on fig. 2 (a).

Step c.2: using 8 pre-defined filtering templates for an image I to be detected

Filtering to obtain 8 filtered images

Step c.3: examining whether any pixel (x, y) in the image I to be detected meets (x, y) epsilon B _i (ii) a Wherein, (x, y) refers to cartesian coordinates of the pixel points; if so, calculate B _i Global entropy E of _i Further obtain the lifting point threshold T _i ＝f(E _i )；

Step c.4: survey vector

If the investigation vector contains 5 continuous elements which are connected end to end and are all larger than T _i Then (x, y) is marked as gFAST feature point; and traversing all pixels to obtain a preliminary gFAST feature point set X' = { X _i ＝(x,y) _i }; fig. 3 (b) shows an example of the extracted adaptive gFAST feature points.

Step c.5: executing an NMS algorithm on the X' to obtain a gFAST key point set X;

on completion of step c, continuing to perform step d: extracting FQPCET characteristics;

the step d specifically comprises the following steps:

step d.1: determining any element X in a set X _i Circular neighborhood O of _i ；

Step d.2: from O _i Obtaining the red, green and blue color channel components f _R [i,j]、f _G [i,j]And f _B [i,j]Wherein, (i, j) refers to pixel point location;

and performing upsampling to obtain f [ u, v ], (u, v) representing the position of the upsampled pixel; :

step d.3: by calculating f u, v]Fast two-dimensional Fourier transform to obtain polar harmonic transform PCET coefficient matrix M _nl ：

Wherein, the first and the second end of the pipe are connected with each other,

representing a fast two-dimensional fourier transform;

step d.4: will f is mixed _R [i,j]、f _G [i,j]And f _B [i,j]Calculating the obtained polar harmonic transformation PCET coefficient matrix

Polar harmonic transform PCET coefficient matrix

Harmonic-polar transformation PCET coefficient matrix

Substituting the formula to obtain a QPCET coefficient matrix of quaternion polar harmonic transformation

Step d.5: QPCET coefficient matrix for quaternion polar harmonic transformation

Selecting characteristics to form a characteristic vector f:

in this case, the characteristic is shown in fig. 3 (c).

Step d.6: traversing the feature point set X and aiming at any feature point X _i Step d.1-step d.5 are performed, get the characteristic to set of quantities F = { F = { F = _i Get the coordinate-feature set S = { S } _i ＝{x _i ,f _i }}；

On completion of step d, continuing to perform step e: performing kNN-based feature matching on the coordinate-feature set S to obtain a matching point set P = { (S, S') _i }; fig. 3 (d) and fig. 3 (e) are schematic diagrams of a matching pair and a correct matching pair, respectively.

On completion of step e, continuing to perform step f: clustering post-processing based on k-means;

the step f specifically comprises the following steps:

step f.1: for matching point set P = { (s, s') _i Performing offset vector calculation to obtain an offset vector set Dt:

Dt＝{(Δx,Δy) _i }＝{(x-x′) _i }; fig. 3 (f) and 3 (g) are schematic diagrams of offset vector cluster 1 and offset vector cluster 2, respectively.

Step f.2: calculating the length and direction of any offset vector in Dt to obtain a length set Le and a direction set Or:

step f.3: carrying out histogram statistics on Le and Or to obtain an offset vector set Dt' after primary screening;

step f.4: performing k-means clustering on Dt' to obtain k classes { C ₁ ,C ₂ ,…,C _k }; fig. 3 (h), fig. 3 (i), fig. 3 (j), and fig. 3 (k) are schematic diagrams of a cluster positioning result 1, a cluster positioning result 2, a cluster positioning result 3, and an overlay result, respectively.

Step f.5: traversing all the clustering results, and clustering C to any one of the clustering results _i RANSAC, ZNCC and morphological processing operations are sequentially executed, and the union of the obtained results is used as a final detection result. Fig. 3 (l) shows the result of marking the resulting tampered area.

It is noted that fig. 4 (a) -4 (d) of the accompanying drawings respectively provide examples of positioning tampered areas of the FAU image library; fig. 5 (a) -5 (d) provide diagrams of examples of tampered area locations of a GRIP image library, respectively. And FIGS. 6 (a) -6 (d) are graphs showing the comparison result of the robustness of the detection method and the comparison method in the FAU image library provided by the embodiment of the invention. Fig. 7 (a) -7 (d) are graphs showing robustness comparison results of the detection method and the comparison method in the GRIP image library according to the embodiment of the present invention. It should be added that, the above detection method is performed by experiments in Matlab R2016a environment, where the two image libraries involved are FAU and GRIP, respectively; the image library is already disclosed on the internet, and a person skilled in the art can search and download the image library by himself to complete the detection process of the disclosed steps.

The invention provides a multi-object tampering detection method based on fast quaternion polar harmonic transformation, which comprises the steps of firstly obtaining a functional relation between an entropy rate and a gFAST characteristic point extraction threshold value through learning, determining an adaptive threshold value by using the function and extracting a gFAST characteristic point; then constructing local features of the image through an FQPCET algorithm; then, obtaining a matching result by using a characteristic matching algorithm based on kNN; and finally, performing k-means clustering on the offset of the matching result, and performing RANSAC, ZNCC and morphological processing on various types to obtain a final positioning result. The experimental result shows that the method not only can effectively detect the multi-copy-paste tampering, but also has good robustness to conventional signal attack and geometric attack, and the time cost of the algorithm is very low.

Compared with the prior art, the invention has the following beneficial effects:

firstly, an entropy-threshold function fitting algorithm is provided and adopted, the method can self-adaptively determine a point-extracting threshold value, and ensure that the global density of characteristic points is uniform, so that the tampering of a small area or a smooth area can be effectively detected;

secondly, a new key point detection method gFAST is provided and adopted, the method is robust to various geometric attacks, the time complexity is small, and the algorithm speed and the geometric attack robustness are improved;

thirdly, a new moment calculation method FQPCET is provided and adopted, the method has remarkable advantages in the aspects of calculation speed, numerical stability and calculation precision compared with the original calculation method, the color information of the image and the correlation among color channels can be effectively reflected, and the detection precision and the algorithm speed of the method are improved.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (5)

1. The multi-object tampering detection method based on the fast quaternion polar harmonic transformation is characterized by comprising the following steps of:

step a: acquiring an image I to be detected, and setting thresholds C =0.1, delta =0.01, N =5 and M =4N; wherein C is an ideal feature point density threshold; Δ is a maximum deviation threshold; n represents the neighborhood diameter of the feature point; m represents the fast quaternion polar harmonic transform FQPCET upsampling parameters;

step b: training to obtain image set (TP) _i An entropy-threshold fitting function T = f (E);

step c: self-adaptive gFAST feature point detection;

step d: extracting FQPCET characteristics;

step e: to the coordinate-the feature set S performs a kNN-based feature matching, obtaining a matching point set P = { (S, S') _i }；

Step f: clustering post-processing based on k-means.

2. The method for multi-object tamper detection based on fast quaternion polar harmonic transform as claimed in claim 1, wherein the step b specifically comprises:

step b.1: executing SLIC algorithm on any image in FAU library to obtain training image set (TP) _i }；

Step b.2: calculating TP _i Entropy E of _i Obtaining an entropy description vector E = { E = } _i }；

Step b.3: extraction of TP _i The characteristic points of (1); solving a point-extracting threshold parameter T when the density X of the characteristic points belongs to (C-delta, C + delta) by utilizing a dichotomy _i Obtaining a threshold value description vector T = { T = { (T) _i }；

Step b.4: function fitting was performed using E and T, resulting in an entropy-threshold fitting function T = f (E).

3. The method for multi-object tamper detection based on fast quaternion polar harmonic transform as claimed in claim 1, wherein the step c specifically comprises:

step c.1: executing SLIC block partitioning algorithm on an image I to be detected to obtain a non-overlapping image block set B = { B = { (B) } _i }；

Step c.2: using 8 pre-defined filtering templates for an image I to be detected

Filtering to obtain 8 filtered images

Step c.3: whether any pixel (x, y) in the image I to be detected meets (x, y) epsilon B or not is examined _i (ii) a Wherein, (x, y) refers to cartesian coordinates of the pixel points; if so, calculate B _i Global entropy E of _i And then getThreshold to lifting point T _i ＝f(E _i )；

Step c.4: survey vector

If the investigation vector contains 5 continuous elements which are connected end to end and are all larger than T _i If yes, marking (x, y) as a gFAST characteristic point; and traversing all the pixels to obtain a preliminary gFAST feature point set X' = { X = } _i ＝(x,y) _i }；

Step c.5: and executing an NMS algorithm on the X' to obtain a gFAST key point set X.

4. The method for multi-object tamper detection based on fast quaternion polar harmonic transform as claimed in claim 1, wherein the step d specifically comprises:

step d.1: determining any element X in a set X _i Circular neighborhood O of _i ；

Step d.2: from O _i Obtaining the red, green and blue color channel components f _R [i,j]、f _G [i,j]And f _B [i,j]Wherein, (i, j) refers to pixel point locations;

and performing upsampling to obtain f [ u, v ], (u, v) representing the position of the upsampled pixel; :

step d.3: by calculating f [ u, v ]]Fast two-dimensional Fourier transform to obtain polar harmonic transform PCET coefficient matrix M _nl ：

Wherein the content of the first and second substances,

representing a fast two-dimensional fourier transform;

step d.4: will f is mixed _R [i,j]、f _G [i,j]And f _B [i,j]Calculating the obtained polar harmonic transformation PCET coefficient matrix

Polar harmonic transformation PCET coefficient matrix

Harmonic-polar transformation PCET coefficient matrix

Substituting the formula to obtain a quaternion polar harmonic transformation QPCET coefficient matrix

Step d.5: QPCET coefficient matrix for quaternion polar harmonic transformation

Selecting characteristics to form a characteristic vector f:

step d.6: traversing the feature point set X and aiming at any feature point X _i Step d.1-step d.5 are performed, get the characteristic to set of quantities F = { F = { F = _i Get the coordinate-feature set S = { S = } _i ＝{x _i ,f _i }}。

5. The method for multi-object tamper detection based on fast quaternion polar harmonic transform as claimed in claim 1, wherein the step f specifically comprises:

step f.1: for matching point set P = { (s, s') _i Performing offset vector calculation to obtain an offset vector set Dt:

Dt＝{(Δx,Δy) _i }＝{(x-x′) _i }；

step f.2: calculating the length and direction of any offset vector in Dt to obtain a length set Le and a direction set Or:

Le＝{||(Δx,Δy) _i || ₂ }，

step f.3: carrying out histogram statistics on Le and Or to obtain an offset vector set Dt' after primary screening;

step f.4: performing k-means clustering on Dt' to obtain k classes { C ₁ ,C ₂ ,…,C _k }；

Step f.5: traversing all the clustering results, and clustering C to any one of the clustering results _i RANSAC, ZNCC and morphological processing operations are sequentially executed, and the union of the obtained results is used as a final detection result.

Images