CN113792789B - Class-activated thermodynamic diagram-based image tampering detection and positioning method and system - Google Patents
Class-activated thermodynamic diagram-based image tampering detection and positioning method and system Download PDFInfo
- Publication number
- CN113792789B CN113792789B CN202111074784.9A CN202111074784A CN113792789B CN 113792789 B CN113792789 B CN 113792789B CN 202111074784 A CN202111074784 A CN 202111074784A CN 113792789 B CN113792789 B CN 113792789B
- Authority
- CN
- China
- Prior art keywords
- image
- tampering
- neural network
- image tampering
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 47
- 238000010586 diagram Methods 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000004913 activation Effects 0.000 claims abstract description 47
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 47
- 238000003379 elimination reaction Methods 0.000 claims abstract description 17
- 230000008030 elimination Effects 0.000 claims abstract description 16
- 230000004807 localization Effects 0.000 claims abstract description 14
- 238000000605 extraction Methods 0.000 claims description 24
- 238000013528 artificial neural network Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 230000002708 enhancing effect Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000003796 beauty Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 101100272279 Beauveria bassiana Beas gene Proteins 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
Abstract
The invention discloses a method and a system for detecting and positioning image tampering based on class activation thermodynamic diagrams, wherein the method comprises the following steps: constructing an object elimination, stitching and copy-paste image tampering dataset by using an image restoration algorithm based on the COCO dataset; performing image tampering detection on the image tampering data set based on the constructed convolutional neural network to obtain an image tampering classification result of the image tampering data set; for the image tampering classification result, a model-based interpretability method finds out pixels in the image affecting the classification result and generates a class activation thermodynamic diagram, and a ResNet50 network is used to generate a final tampering localization area. The invention can more effectively realize tamper detection and positioning for different tamper modes, in particular to detection of tamper elimination modes of objects based on various image restoration algorithms, so that tamper detection and positioning have better generalization capability.
Description
Technical Field
The invention relates to the technical field of multimedia information security, in particular to a method and a system for detecting and positioning image tampering based on class activation thermodynamic diagrams.
Background
With the rapid development and use of multimedia acquisition devices, people have come to the era of multimedia information explosion, and almost everyone has the ability to make and transmit a large number of digital images. Meanwhile, image editing software such as Photoshop, beauty show, beauty camera and the like is simple to operate, so that image modification becomes easier and easier, common people can process and modify images easily, and fake images become less and less noticeable with the development of technology, and even can be in false and spurious. In today's society, the so-called "eye-in-the-eye" has become increasingly unreliable, and when one is faced with an image, there is often first a doubt about the authenticity of the image. In daily life, people modify images usually for the purposes of beautifying and entertainment, and the purposes of beautifying and entertainment are not influenced, but in some cases, the judgment of people on objective things is influenced by the transmission of images which are tampered maliciously, and sometimes adverse effects are caused on society and countries, so that how to accurately and efficiently detect whether digital images are tampered and detect tampered areas has important significance on image evidence collection tasks.
The present tampering modes of image contents mainly comprise splicing, copying-pasting and object elimination based on image restoration, and the operations destroy textures and pattern noise of original images and leave tampering traces, so that whether the images are tampered or not can be judged by comparing inconsistencies of specific fingerprint information in the images or analyzing whether certain statistical features in the images are destroyed or not, and tampered areas are detected. The conventional image tampering detection algorithm can be roughly divided into a conventional algorithm and a deep learning algorithm, and the conventional algorithm is characterized by manually extracting features, establishing a model, analyzing the features and classifying the features, but the conventional method usually adopts a manual design mode to extract the features, and the features based on the manual design have limitations and lack of representativeness, so that multiple tampering modes can not be judged according to the features at the same time; the deep learning algorithm mainly extracts characteristic reclassification by using a convolutional neural network model, and realizes an end-to-end self-adaptive learning mode. However, most of the current image tampering detection algorithms can only detect one or several modification modes, especially the modification mode based on the object elimination mode of image restoration is more difficult with the application of deep learning in the field of image restoration.
Therefore, how to more effectively realize tamper detection and positioning for different tamper modes, and in particular to detect tamper elimination modes of objects based on various image restoration algorithms, so that tamper detection and positioning have better generalization capability is a problem to be solved urgently.
Disclosure of Invention
In view of the above, the invention provides a method for detecting and positioning image tampering based on class activation thermodynamic diagrams, which can more effectively realize the detection and positioning of tampering in different tampering modes, and particularly, the detection of eliminating tampering modes of objects based on various image restoration algorithms, so that the tamper detection and positioning have better generalization capability.
The invention provides a class-activated thermodynamic diagram-based image tampering detection and positioning method, which comprises the following steps:
constructing an object elimination, stitching and copy-paste image tampering dataset by using an image restoration algorithm based on the COCO dataset;
performing image tampering detection on the image tampering data set based on the constructed convolutional neural network to obtain an image tampering classification result of the image tampering data set;
and aiming at the image tampering classification result, finding out pixels influencing the classification result in the image based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering localization area by using a ResNet50 network.
Preferably, the performing image tampering detection on the image tampering dataset by using the convolutional neural network based on construction to obtain an image tampering classification result of the image tampering dataset includes:
performing enhancement processing on the image tampering data set by using the convolutional neural network;
extracting features of the image falsified data set after the enhancement processing;
and deciding a tampered image in the image tampering dataset and a classification result of the tampered image based on the extracted features.
Preferably, the enhancing the image tampering data set by using the convolutional neural network includes:
extracting texture features and edge features by using a convolution layer of the convolution neural network;
extracting noise domain features by using a constraint convolutional layer of the convolutional neural network and an SRM filter;
and fusing the texture features, the edge features and the noise domain features to obtain fusion features.
Preferably, the feature extraction of the image tampered data set after enhancement processing includes:
and performing feature extraction on the fusion features by using 5 convolution layers and batch standardization of the convolution neural network to obtain feature extraction results.
Preferably, the determining the tampered image in the image tampered dataset based on the extracted features and the classification result of the tampered image includes:
based on the feature extraction result, an average pooling layer and a full connection layer of the convolutional neural network are used, and finally, tampered images in the image tampering dataset and classification results of the tampered images are determined through a Softmax activation function.
A system for image tamper detection and localization based on class activation thermodynamic diagrams, comprising:
the construction module is used for constructing an object elimination, splicing and copy-paste image tampering data set by using an image restoration algorithm based on the COCO data set;
the detection module is used for carrying out image tampering detection on the image tampering data set based on the constructed convolutional neural network to obtain an image tampering classification result of the image tampering data set;
and the positioning module is used for finding out pixels affecting the classification result in the image according to the image tampering classification result based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering positioning area by using a ResNet50 network.
Preferably, the detection module includes:
an enhancement unit, configured to perform enhancement processing on the image tampered data set using the convolutional neural network;
the feature extraction unit is used for extracting features of the image falsified data set after the enhancement processing;
and the decision unit is used for deciding the tampered images in the image tampering dataset and the classification results of the tampered images based on the extracted features.
Preferably, the enhancing unit is specifically configured to:
extracting texture features and edge features by using a convolution layer of the convolution neural network;
extracting noise domain features by using a constraint convolutional layer of the convolutional neural network and an SRM filter;
and fusing the texture features, the edge features and the noise domain features to obtain fusion features.
Preferably, the feature extraction unit is specifically configured to:
and performing feature extraction on the fusion features by using 5 convolution layers and batch standardization of the convolution neural network to obtain feature extraction results.
Preferably, the decision unit is specifically configured to:
based on the feature extraction result, an average pooling layer and a full connection layer of the convolutional neural network are used, and finally, tampered images in the image tampering dataset and classification results of the tampered images are determined through a Softmax activation function.
In summary, the invention discloses a method for detecting and positioning image tampering based on class activation thermodynamic diagrams, when tampering detection and tampering position positioning are required to be carried out on an image, an object is eliminated, an image tampering dataset is spliced and copied and pasted by using an image restoration algorithm based on a COCO dataset, then image tampering detection is carried out on the image tampering dataset based on a structured convolutional neural network, and an image tampering classification result of the image tampering dataset is obtained; for the image tampering classification result, a model-based interpretability method finds out pixels in the image affecting the classification result and generates a class activation thermodynamic diagram, and a ResNet50 network is used to generate a final tampering localization area. The invention can more effectively realize tamper detection and positioning for different tamper modes, in particular to detection of tamper elimination modes of objects based on various image restoration algorithms, so that tamper detection and positioning have better generalization capability.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an embodiment 1 of a method for detecting and locating image tampering based on class activation thermodynamic diagrams;
FIG. 2 is a flow chart of embodiment 2 of a method for detecting and locating image tampering based on class activation thermodynamic diagrams;
FIG. 3 is a schematic diagram of a system embodiment 1 for detecting and locating image tampering based on class activation thermodynamic diagrams;
fig. 4 is a schematic structural diagram of an embodiment 2 of a system for detecting and locating image tampering based on class activation thermodynamic diagrams.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a flowchart of an embodiment 1 of a method for detecting and locating image tampering based on class activation thermodynamic diagrams according to the present disclosure may include the following steps:
s101, constructing an object elimination, splicing and copy-paste image tampering data set by using an image restoration algorithm based on a COCO data set;
when the image is required to be tampered and positioned, firstly, the image tampering data set which is erased, spliced and copied and pasted is constructed based on the COCO data set by using an image restoration algorithm. For example, based on the COCO dataset, 10 image restoration algorithms can be used to construct an object-erased, stitched, and copy-pasted image-tampered dataset.
S102, performing image tampering detection on the image tampering data set based on the constructed convolutional neural network to obtain an image tampering classification result of the image tampering data set;
after the image falsification data set is obtained, a convolutional neural network is further constructed, the constructed convolutional neural network is used for carrying out image falsification detection on the image falsification data set, tampered images in the image falsification data set are determined, and tampered classification results of the tampered images are determined. That is, it is determined whether the tampered image belongs to object erasure tampering, splice tampering, or copy-paste tampering.
S103, aiming at the image tampering classification result, finding out pixels influencing the classification result in the image based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering localization area by using a ResNet50 network.
For tampered images determined in the image tampering dataset, aiming at the image tampering classification result of the tampered images, further finding out pixels influencing the classification result in the images based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering positioning area by using a ResNet50 network.
In summary, in the above embodiment, when tampering detection and tampering location positioning are required for an image, an object is constructed by using an image restoration algorithm based on a COCO dataset, and the image tampering dataset is spliced and copied and pasted, and then image tampering detection is performed on the image tampering dataset based on the constructed convolutional neural network, so as to obtain an image tampering classification result of the image tampering dataset; for the image tampering classification result, a model-based interpretability method finds out pixels in the image affecting the classification result and generates a class activation thermodynamic diagram, and a ResNet50 network is used to generate a final tampering localization area. The method can more effectively realize tamper detection and positioning of different tamper modes, and particularly can detect tamper elimination modes of objects based on various image restoration algorithms, so that tamper detection and positioning have better generalization capability.
As shown in fig. 2, a flowchart of an embodiment 2 of a method for detecting and locating image tampering based on class activation thermodynamic diagrams according to the present invention may include the following steps:
s201, constructing an object elimination, splicing and copy-paste image tampering data set by using an image restoration algorithm based on the COCO data set;
when the image is required to be tampered and positioned, firstly, the image tampering data set which is erased, spliced and copied and pasted is constructed based on the COCO data set by using an image restoration algorithm. For example, based on the COCO dataset, 10 image restoration algorithms can be used to construct an object-erased, stitched, and copy-pasted image-tampered dataset.
S202, enhancing the image falsification data set by using a convolutional neural network;
after the image falsification data set is obtained, a convolutional neural network is further constructed, and the constructed convolutional neural network is used for enhancing the image falsification data set.
Specifically, when the convolutional neural network is used for enhancing the image falsification data set, a convolutional layer of the convolutional neural network is used for extracting texture features and edge features, and a constraint convolutional layer of the convolutional neural network and an SRM filter are used for extracting noise domain features; and then fusing texture features, edge features and noise domain features to obtain fused features.
The construction of the constraint convolution layer of the convolution neural network comprises the following specific steps:
randomly initializing a first layer of the convolutional neural network, and constraining a convolutional kernel based on the following expression:
wherein,representing the pixel points in the convolution kernel (m,n) weight of position ∈>A value representing the position of the center pixel point of the convolution kernel. The weight value of the central pixel point of the constraint convolution kernel is-1, and the sum of the weights of surrounding pixel points is 1.
Specifically, the SRM filter is mainly composed of 3 convolution kernels:
s203, extracting features of the image falsified data set after the enhancement processing;
and then, extracting the characteristics of the image falsification data set after the enhancement processing.
Specifically, the fusion features are subjected to feature extraction by using 5 convolution layers of a convolution neural network and batch standardization, and a feature extraction result is obtained.
S204, deciding a tampered image in the image tampering dataset based on the extracted features, and classifying results of the tampered image;
then, a tampered image in the image tampering dataset and a classification result of the tampered image are decided based on the extracted features.
Specifically, based on the feature extraction result, an average pooling layer and a full connection layer of the convolutional neural network are used, and finally, tampered images in the image tampered data set and classification results of the tampered images are decided through a Softmax activation function.
S205, aiming at the image tampering classification result, finding out pixels influencing the classification result in the image based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering localization area by using a ResNet50 network.
The method based on model interpretability finds out pixels in the image affecting the classification result and generates a class activation thermodynamic diagram, and generates a final tamper localization area by using a ResNet50 network, which specifically comprises the following steps:
(1) Calculate the last layerTamper class probability y in Softmax activation function output c For all pixels A of the final layer of the feature map i,j The partial derivatives of (a), namely:
wherein y is a probability vector output by a Softmax activation function, c is a sequence number of a tampered class, A is a feature diagram output by a convolution layer of the last layer, k is a sequence number of a channel dimension of the feature diagram, and i and j are sequence numbers of a wide dimension and a high dimension respectively.
(2) Will y c Taking an average of partial derivatives of each pixel of the feature map in a wide and high dimension to obtain sensitivity of the tampering class to a kth channel of the feature map output by a final layer of convolution layer:
(3) Will beAs weights, the weights are combined linearly with the final layer of feature map, and are processed by using the Relu activation function to obtain a class activation thermodynamic diagram:
(4) A tamper localization area of the final tamper area is generated by the res net50 network.
In summary, the detection and positioning of the tampered image are realized through the convolutional neural network model and the class activation thermodynamic diagram. The invention has the advantages of simpler model structure and better model generalization performance, and can be suitable for detection and positioning scenes of image splicing, copy-pasting and object tamper elimination modes.
Referring to fig. 3, a schematic structural diagram of an embodiment 1 of a system for detecting and locating image tampering based on class activation thermodynamic diagrams according to the present disclosure may include:
a construction module 301 for constructing an object-elimination, stitching, and copy-paste image-tampered dataset using an algorithm for image restoration based on the COCO dataset;
when the image is required to be tampered and positioned, firstly, the image tampering data set which is erased, spliced and copied and pasted is constructed based on the COCO data set by using an image restoration algorithm. For example, based on the COCO dataset, 10 image restoration algorithms can be used to construct an object-erased, stitched, and copy-pasted image-tampered dataset.
The detection module 302 is configured to perform image tampering detection on the image tampering dataset based on the constructed convolutional neural network, so as to obtain an image tampering classification result of the image tampering dataset;
after the image falsification data set is obtained, a convolutional neural network is further constructed, the constructed convolutional neural network is used for carrying out image falsification detection on the image falsification data set, tampered images in the image falsification data set are determined, and tampered classification results of the tampered images are determined. That is, it is determined whether the tampered image belongs to object erasure tampering, splice tampering, or copy-paste tampering.
The positioning module 303 is configured to find out pixels affecting the classification result in the image and generate a class activation thermodynamic diagram according to the model interpretability-based method for the image tampering classification result, and generate a final tampering positioning area by using the ResNet50 network.
For tampered images determined in the image tampering dataset, aiming at the image tampering classification result of the tampered images, further finding out pixels influencing the classification result in the images based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering positioning area by using a ResNet50 network.
In summary, in the above embodiment, when tampering detection and tampering location positioning are required for an image, an object is constructed by using an image restoration algorithm based on a COCO dataset, and the image tampering dataset is spliced and copied and pasted, and then image tampering detection is performed on the image tampering dataset based on the constructed convolutional neural network, so as to obtain an image tampering classification result of the image tampering dataset; for the image tampering classification result, a model-based interpretability method finds out pixels in the image affecting the classification result and generates a class activation thermodynamic diagram, and a ResNet50 network is used to generate a final tampering localization area. The method can more effectively realize tamper detection and positioning of different tamper modes, and particularly can detect tamper elimination modes of objects based on various image restoration algorithms, so that tamper detection and positioning have better generalization capability.
Referring to fig. 4, a schematic structural diagram of an embodiment 2 of a system for detecting and locating image tampering based on class activation thermodynamic diagrams according to the present disclosure may include:
a construction module 401 for constructing an object-erased, stitched, and copy-pasted image-tampered dataset using an algorithm for image restoration based on the COCO dataset;
when the image is required to be tampered and positioned, firstly, the image tampering data set which is erased, spliced and copied and pasted is constructed based on the COCO data set by using an image restoration algorithm. For example, based on the COCO dataset, 10 image restoration algorithms can be used to construct an object-erased, stitched, and copy-pasted image-tampered dataset.
An enhancement unit 402, configured to perform enhancement processing on the image tampered data set using a convolutional neural network;
after the image falsification data set is obtained, a convolutional neural network is further constructed, and the constructed convolutional neural network is used for enhancing the image falsification data set.
Specifically, when the convolutional neural network is used for enhancing the image falsification data set, a convolutional layer of the convolutional neural network is used for extracting texture features and edge features, and a constraint convolutional layer of the convolutional neural network and an SRM filter are used for extracting noise domain features; and then fusing texture features, edge features and noise domain features to obtain fused features.
The construction of the constraint convolution layer of the convolution neural network comprises the following specific steps:
randomly initializing a first layer of the convolutional neural network, and constraining a convolutional kernel based on the following expression:
wherein,weights representing the position of pixel (m, n) in the convolution kernel, +.>A value representing the position of the center pixel point of the convolution kernel. The weight value of the central pixel point of the constraint convolution kernel is-1, and the sum of the weights of surrounding pixel points is 1.
Specifically, the SRM filter is mainly composed of 3 convolution kernels:
a feature extraction unit 403, configured to perform feature extraction on the image tampered data set after the enhancement processing;
and then, extracting the characteristics of the image falsification data set after the enhancement processing.
Specifically, the fusion features are subjected to feature extraction by using 5 convolution layers of a convolution neural network and batch standardization, and a feature extraction result is obtained.
A decision unit 404, configured to decide, based on the extracted features, a tampered image in the image tampered dataset, and a classification result of the tampered image;
then, a tampered image in the image tampering dataset and a classification result of the tampered image are decided based on the extracted features.
Specifically, based on the feature extraction result, an average pooling layer and a full connection layer of the convolutional neural network are used, and finally, tampered images in the image tampered data set and classification results of the tampered images are decided through a Softmax activation function.
The positioning module 405 is configured to find out pixels affecting the classification result in the image and generate a class activation thermodynamic diagram according to the model interpretability-based method, and generate a final tampered positioning area using the res net50 network.
The method based on model interpretability finds out pixels in the image affecting the classification result and generates a class activation thermodynamic diagram, and generates a final tamper localization area by using a ResNet50 network, which specifically comprises the following steps:
(1) Calculating falsification class probability y in output of final layer Softmax activation function c For all pixels A of the final layer of the feature map i,j The partial derivatives of (a), namely:
wherein y is a probability vector output by a Softmax activation function, c is a sequence number of a tampered class, A is a feature diagram output by a convolution layer of the last layer, k is a sequence number of a channel dimension of the feature diagram, and i and j are sequence numbers of a wide dimension and a high dimension respectively.
(2) Will y c Taking an average of partial derivatives of each pixel of the feature map in a wide and high dimension to obtain sensitivity of the tampering class to a kth channel of the feature map output by a final layer of convolution layer:
(3) Will beWeighted by the weight and the last layer of characteristic diagram, are linearly combined and use the Relu activation functionAnd (3) performing number processing to obtain a class activation thermodynamic diagram:
(4) A tamper localization area of the final tamper area is generated by the res net50 network.
In summary, the detection and positioning of the tampered image are realized through the convolutional neural network model and the class activation thermodynamic diagram. The invention has the advantages of simpler model structure and better model generalization performance, and can be suitable for detection and positioning scenes of image splicing, copy-pasting and object tamper elimination modes.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (2)
1. A method for detecting and locating image tampering based on class activation thermodynamic diagrams, comprising:
constructing an object elimination, stitching and copy-paste image tampering dataset by using an image restoration algorithm based on the COCO dataset;
performing image tampering detection on the image tampering data set based on the constructed convolutional neural network to obtain an image tampering classification result of the image tampering data set;
the method for detecting the image tampering of the image tampering data set based on the structured convolutional neural network comprises the following steps of: extracting texture features and edge features by using a convolution layer of the convolution neural network; extracting noise domain features by using a constraint convolutional layer of the convolutional neural network and an SRM filter; fusing the texture features, the edge features and the noise domain features to obtain fused features; performing feature extraction on the fusion features by using 5 convolution layers and batch standardization of the convolution neural network to obtain feature extraction results; based on the feature extraction result, an average pooling layer and a full connection layer of the convolutional neural network are used, and finally, tampered images in the image tampering dataset and classification results of the tampered images are determined through a Softmax activation function;
and aiming at the image tampering classification result, finding out pixels influencing the classification result in the image based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering localization area by using a ResNet50 network.
2. A system for image tamper detection and localization based on class activation thermodynamic diagrams, comprising:
the construction module is used for constructing an object elimination, splicing and copy-paste image tampering data set by using an image restoration algorithm based on the COCO data set;
the detection module is used for carrying out image tampering detection on the image tampering data set based on the constructed convolutional neural network to obtain an image tampering classification result of the image tampering data set;
wherein, the detection module includes: an enhancement unit for extracting texture features and edge features using a convolutional layer of the convolutional neural network; extracting noise domain features using a constrained convolutional layer of the convolutional neural network and an SRM filter; the texture features, the edge features and the noise domain features are fused to obtain fusion features; the feature extraction unit is used for carrying out feature extraction on the fusion features by using 5 convolution layers and batch standardization of the convolution neural network to obtain feature extraction results; the decision unit is used for deciding the tampered images in the image tampering dataset and the classification results of the tampered images by using an average pooling layer and a full connection layer of the convolutional neural network based on the feature extraction result and finally by using a Softmax activation function;
and the positioning module is used for finding out pixels affecting the classification result in the image according to the image tampering classification result based on a model interpretability method, generating a class activation thermodynamic diagram, and generating a final tampering positioning area by using a ResNet50 network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111074784.9A CN113792789B (en) | 2021-09-14 | 2021-09-14 | Class-activated thermodynamic diagram-based image tampering detection and positioning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111074784.9A CN113792789B (en) | 2021-09-14 | 2021-09-14 | Class-activated thermodynamic diagram-based image tampering detection and positioning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113792789A CN113792789A (en) | 2021-12-14 |
| CN113792789B true CN113792789B (en) | 2024-03-29 |
Family
ID=78880295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111074784.9A Active CN113792789B (en) | 2021-09-14 | 2021-09-14 | Class-activated thermodynamic diagram-based image tampering detection and positioning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113792789B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349136A (en) * | 2019-06-28 | 2019-10-18 | 厦门大学 | A kind of tampered image detection method based on deep learning |
| CN112215780A (en) * | 2020-10-28 | 2021-01-12 | 浙江工业大学 | Image evidence-obtaining antagonistic attack defense method based on class feature restoration and fusion |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10902543B2 (en) * | 2018-03-15 | 2021-01-26 | Tata Consultancy Services Limited | Neural network based insertion of watermark into images and tampering detection thereof |
-
2021
- 2021-09-14 CN CN202111074784.9A patent/CN113792789B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349136A (en) * | 2019-06-28 | 2019-10-18 | 厦门大学 | A kind of tampered image detection method based on deep learning |
| CN112215780A (en) * | 2020-10-28 | 2021-01-12 | 浙江工业大学 | Image evidence-obtaining antagonistic attack defense method based on class feature restoration and fusion |
Non-Patent Citations (2)
| Title |
|---|
| 朱新山 ; 钱永军 ; 孙彪 ; 任超 ; 孙亚 ; 姚思如 ; .基于深度神经网络的图像修复取证算法.光学学报.2018,(11),全文. * |
| 陈鹏 ; 梁涛 ; 刘锦 ; 戴娇 ; 韩冀中 ; .融合全局时序和局部空间特征的伪造人脸视频检测方法.信息安全学报.2020,(02),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113792789A (en) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zheng et al. | A survey on image tampering and its detection in real-world photos | |
| Xiao et al. | Image splicing forgery detection combining coarse to refined convolutional neural network and adaptive clustering | |
| Bi et al. | RRU-Net: The ringed residual U-Net for image splicing forgery detection | |
| Zeng et al. | Image splicing localization using PCA-based noise level estimation | |
| Asghar et al. | Copy-move and splicing image forgery detection and localization techniques: a review | |
| Abdel-Basset et al. | 2-Levels of clustering strategy to detect and locate copy-move forgery in digital images | |
| Guo et al. | Fake colorized image detection | |
| Liu et al. | A passive image authentication scheme for detecting region-duplication forgery with rotation | |
| Korus et al. | Multi-scale fusion for improved localization of malicious tampering in digital images | |
| Pan et al. | Exposing image forgery with blind noise estimation | |
| Hegazi et al. | An improved copy-move forgery detection based on density-based clustering and guaranteed outlier removal | |
| Xiao et al. | Defocus blur detection based on multiscale SVD fusion in gradient domain | |
| Thajeel et al. | A Novel Approach for Detection of Copy Move Forgery using Completed Robust Local Binary Pattern. | |
| CN112561907B (en) | Video tampering operation detection method and device based on double-current network | |
| Steinebach et al. | Fake news detection by image montage recognition | |
| Chakraborty | PRNU-based image manipulation localization with discriminative random fields | |
| Zhang et al. | Distinguishing photographic images and photorealistic computer graphics using visual vocabulary on local image edges | |
| Tahaoglu et al. | Improved copy move forgery detection method via L* a* b* color space and enhanced localization technique | |
| Nawaz et al. | Single and multiple regions duplication detections in digital images with applications in image forensic | |
| Zhang et al. | Multi-scale segmentation strategies in PRNU-based image tampering localization | |
| Xue et al. | JPEG image tampering localization based on normalized gray level co-occurrence matrix | |
| Meena et al. | Image splicing forgery detection using noise level estimation | |
| CN116563565B (en) | Digital image tampering identification and source region and target region positioning method based on field adaptation, computer equipment and storage medium | |
| CN113792789B (en) | Class-activated thermodynamic diagram-based image tampering detection and positioning method and system | |
| CN115223022B (en) | Image processing method, device, storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |