CN114444565B - Image tampering detection method, terminal equipment and storage medium - Google Patents
Image tampering detection method, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN114444565B CN114444565B CN202111533371.2A CN202111533371A CN114444565B CN 114444565 B CN114444565 B CN 114444565B CN 202111533371 A CN202111533371 A CN 202111533371A CN 114444565 B CN114444565 B CN 114444565B
- Authority
- CN
- China
- Prior art keywords
- image
- loss function
- resampling
- layer
- input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 44
- 238000012952 Resampling Methods 0.000 claims abstract description 29
- 238000011176 pooling Methods 0.000 claims abstract description 16
- 238000005070 sampling Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 37
- 238000000034 method Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 19
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 2
- 238000002372 labelling Methods 0.000 claims description 2
- 238000013135 deep learning Methods 0.000 abstract description 6
- 238000000605 extraction Methods 0.000 abstract description 3
- 238000005242 forging Methods 0.000 abstract 1
- 239000011159 matrix material Substances 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
Abstract
In an image tampering detection model, a network backbone part adopts a Faster-RCNN network, the output of an RPN layer and an input image are input into a new ROI pooling layer together, an image output by a ROI pooling layer is subjected to resampling treatment to obtain a resampling spectrogram, the resampling spectrogram is input into an LSTM network to obtain a resampling feature map, and the output of the resampling feature map after passing through an up-sampling layer and a convolution layer is input into a bilinear pooling layer together with the output of the Faster-RCNN network to be fused. The invention solves the problem that the feature extraction of the current deep learning network in the forging and falsifying field is insufficient.
Description
Technical Field
The present invention relates to the field of image detection, and in particular, to an image tampering detection method, a terminal device, and a storage medium.
Background
With the development of technology and the popularization of software, PS has become increasingly simple and convenient to operate to forge pictures with high quality and reality. Many high quality P-maps not only can fool the human eye, but also can successfully fool some detection models, which poses a great hazard to the development of social security.
The image PS manipulation basic operations can be divided into four types, i.e., copy-move, split-splice (splicing), remove pixels and fill in with other pixels (remove), and generate characters (generated), respectively. In order to increase the fidelity of the forged image, the falsifier not only uses tools to carry out edge feathering, painting brush smearing, color change and other operations, but also carries out noise adding, blurring adding, compression and other post-processing operations. These operations make the image features of the counterfeit area indistinguishable from those of the non-tampered area, increasing the difficulty of detection to some extent.
The PS tampering identification and positioning method is mainly divided into a traditional identification method and a method based on deep learning. The traditional detection and positioning algorithm is an algorithm for detecting and positioning based on the statistical characteristics of images according to the artificial construction characteristics of the user, such as a DCT (discrete cosine transform) falsification detection algorithm, a SIFT (scale-invariant feature transform) detection algorithm, a resampling trace detection algorithm and the like, and the detection and positioning algorithm based on deep learning autonomously learns related characteristics so as to detect PS falsified image areas.
The detection positioning method based on deep learning has been developed for a period of time, but along with the development of image tampering technology, the existing tampering detection algorithm also has some defects, such as that most tampering detection positioning algorithms only extract features based on RGB format images, and the extracted features are limited, so that images of fake areas and non-tampered areas are difficult to distinguish.
Disclosure of Invention
In order to solve the problems, the invention provides an image tampering detection method, a terminal device and a storage medium.
The specific scheme is as follows:
an image tamper detection method comprising the steps of:
S1: acquiring tampered and untampered images to form a training set, and marking tampered areas in the tampered images in the training set;
S2: constructing an image tampering detection model, and training the model through a training set;
The method comprises the steps that a trunk part in a network structure of an image tampering detection model adopts a Faster-RCNN network, an input image is input into the Faster-RCNN network, a candidate frame output by an RPN layer in the Faster-RCNN network is input into a ROI pooling layer in the Faster-RCNN network on one hand, and is input into a new ROI pooling layer together with the input image on the other hand, an image output by the ROI pooling layer is subjected to resampling treatment to obtain a resampling spectrogram, the resampling spectrogram is input into the LSTM network to obtain a resampling feature map, the output of the resampling feature map after being subjected to an up-sampling layer and a convolution layer is input into a bilinear pooling layer together with the output of the Faster-RCNN network to be fused, and a prediction result of whether the image is tampered or not is obtained after the fused result passes through a full-connection layer and a softmax layer;
s3: and detecting whether the image is tampered or not through the trained model.
Further, the resampling process comprises the following steps: and processing the candidate frame region in the image through an EM algorithm, and then carrying out Fourier transformation to obtain a resampling spectrogram.
Further, the calculation formula of the loss function L total of the image tampering detection model is as follows:
Ltotal＝LRPN+Ltamper+Lbbox
where L RPN represents the RPN layer loss function, L tamper represents the final classification loss function, and L bbox represents the rectangular regression loss function.
Further, the loss function of the RPN layer includes a classification loss function L cls and a regression loss function L reg, and the calculation formula of the loss function of the RPN layer is:
Where g i denotes the prediction classification probability of the candidate box, Representing the true label of the candidate box, f i representing the predicted coordinates of the candidate box,The true coordinates of the labeling frames corresponding to the candidate frames are represented, i represents the serial numbers of the candidate frames, lambda represents the weight coefficient, N cls represents the size of the small batch, and N reg represents the number of the candidate frames.
Further, the final classification loss function uses a cross entropy loss function, and the rectangular box regression loss function uses SmoothL loss functions.
An image tamper detection terminal device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to the embodiment of the invention are realized when the processor executes the computer program.
A computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method described above for embodiments of the present invention.
According to the technical scheme, the resampling characteristic is used in the LSTM, the connection between the tampered block and the untampered block is explored, and the problem that the characteristic extraction of the current deep learning network in the fake tampered field is insufficient is solved.
Drawings
Fig. 1 is a flowchart of a first embodiment of the present invention.
Fig. 2 is a diagram showing a network structure of an image falsification detection model in this embodiment.
Detailed Description
For further illustration of the various embodiments, the invention is provided with the accompanying drawings. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments and together with the description, serve to explain the principles of the embodiments. With reference to these matters, one of ordinary skill in the art will understand other possible embodiments and advantages of the present invention.
The invention will now be further described with reference to the drawings and detailed description.
Embodiment one:
the embodiment of the invention provides an image tampering detection method, as shown in fig. 1, comprising the following steps:
S1: and acquiring tampered and untampered images to form a training set, and marking tampered areas in the tampered images in the training set.
S2: and constructing an image tampering detection model, and training the model through a training set.
As shown in FIG. 2, in the network structure of the image tamper detection model, a fast-RCNN network is adopted in a trunk part, after an input image is input into the fast-RCNN network, a candidate frame output by an RPN layer in the fast-RCNN network is input into a ROI pooling layer in the fast-RCNN network on one hand so as to finish the part of the original fast-RCNN network, namely, the coordinates of the candidate frame are corresponding to a feature map through the ROI pooling layer, and a new ROI pooling layer is input together with the input image on the other hand so as to correspond the coordinates of the candidate frame to the input image, the image output by the ROI pooling layer is subjected to resampling processing to obtain a resampling spectrogram, the resampling spectrogram is input into the LSTM network to obtain a resampling feature map, the output of the resampling feature map after the upsampling spectrogram is subjected to upsampling layer and the convolution layer is input into a bilinear pooling layer together with the output of the fast-RCNN network, and the fusion result is subjected to a full-connection layer and a softmax layer so as to obtain a prediction result of whether the image is tampered.
The input image in this embodiment is an RGB image.
The resampling process in this embodiment is as follows: and processing the candidate frame region in the image through an EM algorithm, and then carrying out Fourier transformation to obtain a resampling spectrogram. The specific process is as follows:
1. First, it is assumed that each sampling point belongs to two models, the first model is an M1 model, and the M1 model represents that the point y i currently under investigation has a linear correlation with its neighboring points, that is, the points are generated according to the following model:
Where n (i) is a gaussian distribution of independent co-distributed zero mean unknown variance σ. The second model is the M2 model, which represents that the sample points under investigation do not have a linear correlation with their neighbors, i.e. they are generated by a separate process.
2. The EM algorithm is a two-step iterative algorithm: (1) step E: the probability of the model to which each sampling point belongs is estimated first; (2) M steps: the particular form of correlation of the sampling points is again estimated. The probability of the M1 model to which the sampling point y i belongs can be obtained by a Bayesian formula:
In the above formula, pr { y i∈M1 } and Pr { y i∈M2 } are prior probabilities, the initial value is set to 1/2, and the probability that y i belongs to the M1 model can be expressed as:
In step M, the estimation of the weight vector is calculated by the following formula:
(3) The molecular term in the formula can be expressed as a predicted residual error, namely:
After the calculation of step E is completed, the weight vector in step M is calculated The calculation formula of (2) is as follows:
Wherein the row vector of the Y matrix is a vector composed of 2 N adjacent points of each point in the original image matrix, W is a diagonal matrix, the value on the diagonal of the matrix is the value of omega (i), and the weight vector is calculated by the formula And E and M steps of iterative iteration are carried out until convergence, and a final weight vector can be obtained at the moment.
3. And then carrying out Fourier transform on the posterior probability map to obtain a resample spectrogram, wherein the Fourier transform aims at taking out high-frequency components with periodicity and removing low-frequency components through a high-pass filter. After the fourier transform, if the image contains periodic related information, the fourier transform map contains periodic bright spots.
The calculation formula of the loss function L total of the image tampering detection model is as follows:
Ltotal＝LRPN+Ltamper+Lbbox
where L RPN represents the RPN layer loss function, L tamper represents the final classification loss function, and L bbox represents the rectangular regression loss function.
The RPN layer has two tasks, which candidate frames contain picture main bodies and which only contain backgrounds, namely classification tasks, are judged from a plurality of candidate frames; for the candidate box containing the subject, the regression obtains a real target, namely a regression task, so in this embodiment, the loss of the RPN layer is set to include a classification loss function L cls and a regression loss function L reg, and the calculation formula of the loss function of the RPN layer is as follows:
Where g i denotes the prediction classification probability of the candidate box (anchor), A real label representing a candidate frame (anchor), f i represents coordinates predicted by the candidate frame (anchor),The true coordinates of the label frame (ground truth) corresponding to the candidate frame (anchor) are represented, i represents the number of the candidate frame, λ represents the weight coefficient, N cls represents the size of the small batch (mini-batch size), and N reg represents the number of the candidate frames.
The final classification loss function adopts a cross entropy loss function, and the rectangular box regression loss function adopts SmoothL loss functions.
S3: and detecting whether the image is tampered or not through the trained model.
After the image to be detected is input into the trained image falsification detection model, the model outputs whether the image to be detected is a falsified image or not.
The method provided by the embodiment of the invention is beneficial to solving the problem that the existing deep learning method is insufficient in extraction of the falsified features of the PS falsified image, and the RGB features and the features of resampling traces of the falsified region are fused in the target detection network, so that the falsification detection and positioning capability of the PS image is effectively improved.
Embodiment two:
The invention also provides an image tampering detection terminal device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the steps in the method embodiment of the first embodiment of the invention are realized when the processor executes the computer program.
Further, as an executable scheme, the image tamper detection terminal device may be a computing device such as a desktop computer, a notebook computer, a palm computer, and a cloud server. The image tamper detection terminal device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that the above-described constituent structure of the image tampering detecting terminal device is merely an example of the image tampering detecting terminal device, and does not constitute limitation of the image tampering detecting terminal device, and may include more or less components than those described above, or may combine some components, or different components, for example, the image tampering detecting terminal device may further include an input/output device, a network access device, a bus, etc., which is not limited by the embodiment of the present invention.
Further, as an executable scheme, the Processor may be a central processing unit (Central Processing Unit, CPU), other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, which is a control center of the image tamper detection terminal device, and connects the respective parts of the entire image tamper detection terminal device using various interfaces and lines.
The memory may be used to store the computer program and/or the module, and the processor may implement various functions of the image tamper detection terminal device by running or executing the computer program and/or the module stored in the memory and calling the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created according to the use of the cellular phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart memory card (SMART MEDIA CARD, SMC), secure Digital (SD) card, flash memory card (FLASH CARD), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
The present invention also provides a computer readable storage medium storing a computer program which when executed by a processor implements the steps of the above-described method of an embodiment of the present invention.
The modules/units integrated in the image tamper detection terminal device may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a software distribution medium, and so forth.
While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. An image tamper detection method, comprising the steps of:
S1: acquiring tampered and untampered images to form a training set, and marking tampered areas in the tampered images in the training set;
S2: constructing an image tampering detection model, and training the model through a training set;
The method comprises the steps that a trunk part in a network structure of an image tampering detection model adopts a Faster-RCNN network, an input image is input into the Faster-RCNN network, a candidate frame output by an RPN layer in the Faster-RCNN network is input into a first ROI pooling layer in the Faster-RCNN network on one hand, the candidate frame and the input image are input into a second ROI pooling layer together, an image output by the second ROI pooling layer is subjected to resampling processing to obtain a resampling spectrogram, the resampling spectrogram is input into the LSTM network to obtain a resampling feature map, the output of the resampling feature map after being subjected to an up-sampling layer and a convolution layer is input into a bilinear pooling layer together with the output of the Faster-RCNN network to be fused, and a prediction result of whether the image is a tampered image is obtained after the fusion result passes through a full connection layer and a softmax layer; the resampling process comprises the following steps: processing the candidate frame region in the image through an EM algorithm, and then carrying out Fourier transformation to obtain a resampling spectrogram;
s3: and detecting whether the image is tampered or not through the trained model.
2. The image tampering detection method as defined in claim 1, wherein: the calculation formula of the loss function L total of the image tampering detection model is as follows:
Ltotal＝LRPN+Ltamper+Lbbox
where L RPN represents the RPN layer loss function, L tamper represents the final classification loss function, and L bbox represents the rectangular regression loss function.
3. The image tampering detection method as defined in claim 2, wherein: the loss function of the RPN layer comprises a classification loss function L cls and a regression loss function L reg, and the calculation formula of the loss function of the RPN layer is as follows:
Where g i denotes the prediction classification probability of the candidate box, Representing the true label of the candidate box, f i representing the predicted coordinates of the candidate box,The true coordinates of the labeling frames corresponding to the candidate frames are represented, i represents the serial numbers of the candidate frames, lambda represents the weight coefficient, N cls represents the size of the small batch, and N reg represents the number of the candidate frames.
4. The image tampering detection method as defined in claim 2, wherein: the final classification loss function adopts a cross entropy loss function, and the rectangular box regression loss function adopts SmoothL loss functions.
5. An image tamper detection terminal device, characterized in that: comprising a processor, a memory and a computer program stored in the memory and running on the processor, which processor, when executing the computer program, carries out the steps of the method according to any one of claims 1 to 4.
6. A computer-readable storage medium storing a computer program, characterized in that: the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111533371.2A CN114444565B (en) | 2021-12-15 | Image tampering detection method, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111533371.2A CN114444565B (en) | 2021-12-15 | Image tampering detection method, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114444565A CN114444565A (en) | 2022-05-06 |
| CN114444565B true CN114444565B (en) | 2024-09-24 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110992238A (en) * | 2019-12-06 | 2020-04-10 | 上海电力大学 | Digital image tampering blind detection method based on dual-channel network |
| CN111311563A (en) * | 2020-02-10 | 2020-06-19 | 北京工业大学 | Image tampering detection method based on multi-domain feature fusion |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110992238A (en) * | 2019-12-06 | 2020-04-10 | 上海电力大学 | Digital image tampering blind detection method based on dual-channel network |
| CN111311563A (en) * | 2020-02-10 | 2020-06-19 | 北京工业大学 | Image tampering detection method based on multi-domain feature fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Izidio et al. | An embedded automatic license plate recognition system using deep learning | |
| Jiang et al. | Robust feature matching for remote sensing image registration via linear adaptive filtering | |
| Liu et al. | Locating splicing forgery by fully convolutional networks and conditional random field | |
| CN106683119B (en) | Moving vehicle detection method based on aerial video image | |
| CN111680690B (en) | Character recognition method and device | |
| EP4085369A1 (en) | Forgery detection of face image | |
| CN112381775A (en) | Image tampering detection method, terminal device and storage medium | |
| CN108629286B (en) | Remote sensing airport target detection method based on subjective perception significance model | |
| CN112329702B (en) | Method and device for rapid face density prediction and face detection, electronic equipment and storage medium | |
| CN109086811A (en) | Multi-tag image classification method, device and electronic equipment | |
| Hou et al. | BSNet: Dynamic hybrid gradient convolution based boundary-sensitive network for remote sensing image segmentation | |
| CN112052845A (en) | Image recognition method, device, equipment and storage medium | |
| CN111079764A (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN111104941B (en) | Image direction correction method and device and electronic equipment | |
| CN113449784A (en) | Image multi-classification method, device, equipment and medium based on prior attribute map | |
| CN112101386A (en) | Text detection method and device, computer equipment and storage medium | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN111179270A (en) | Image co-segmentation method and device based on attention mechanism | |
| CN115578590A (en) | Image identification method and device based on convolutional neural network model and terminal equipment | |
| CN115909378A (en) | Document text detection model training method and document text detection method | |
| Qin et al. | Face inpainting network for large missing regions based on weighted facial similarity | |
| WO2024174726A1 (en) | Handwritten and printed text detection method and device based on deep learning | |
| Qu et al. | Deep learning-driven surveillance quality enhancement for maritime management promotion under low-visibility weathers | |
| CN114511702A (en) | Remote sensing image segmentation method and system based on multi-scale weighted attention | |
| CN114444565B (en) | Image tampering detection method, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |